Nov 03 19:24:08.710878 ip-10-0-143-212 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 03 19:24:08.710890 ip-10-0-143-212 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 03 19:24:08.710900 ip-10-0-143-212 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 03 19:24:08.711161 ip-10-0-143-212 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 03 19:24:18.212169 ip-10-0-143-212 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 03 19:24:18.212181 ip-10-0-143-212 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 898a0088c28346cdb9934b4e8869bda5 --
Nov 03 19:25:57.944293 ip-10-0-143-212 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 19:25:58.364774 ip-10-0-143-212 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 19:25:58.364774 ip-10-0-143-212 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 19:25:58.364774 ip-10-0-143-212 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 19:25:58.364774 ip-10-0-143-212 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 19:25:58.368067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.367974    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:25:58.368067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368066    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:25:58.368067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368070    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368073    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368077    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368080    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368082    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368085    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368088    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368090    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368093    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368096    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368098    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368101    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368104    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368107    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368113    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368116    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368118    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368121    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368124    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:25:58.368155 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368126    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368129    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368131    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368134    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368137    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368139    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368142    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368145    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368147    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368150    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368153    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368155    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368160    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368165    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368168    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368171    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368174    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368176    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368179    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:25:58.368591 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368182    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368184    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368187    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368190    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368194    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368198    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368201    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368204    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368206    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368209    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368212    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368214    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368217    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368220    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368223    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368226    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368229    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368231    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368234    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:25:58.369065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368236    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368239    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368242    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368244    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368247    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368250    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368252    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368255    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368257    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368259    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368262    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368264    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368267    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368270    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368273    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368276    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368278    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368281    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368283    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368286    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:25:58.369598 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368288    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368290    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368293    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368295    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368298    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368300    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368303    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368305    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368309    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368312    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368315    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368317    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368320    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368322    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368325    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368327    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368330    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368332    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368335    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368338    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:25:58.370086 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368340    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368342    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368345    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368349    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368351    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368354    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.368357    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369128    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369136    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369139    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369141    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369144    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369147    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369150    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369152    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369155    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369158    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369160    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369163    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:25:58.370593 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369165    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369169    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369172    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369174    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369177    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369179    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369182    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369184    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369187    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369190    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369192    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369195    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369198    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369200    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369203    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369205    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369209    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369211    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369214    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369216    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:25:58.371067 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369219    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369222    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369225    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369227    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369230    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369232    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369237    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369241    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369245    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369249    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369252    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369256    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369259    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369263    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369265    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369268    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369271    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369274    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369277    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:25:58.371547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369280    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369283    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369285    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369288    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369291    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369294    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369297    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369300    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369303    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369305    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369308    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369310    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369313    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369315    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369318    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369321    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369324    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369326    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369329    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:25:58.371992 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369332    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369335    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369337    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369340    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369342    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369345    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369347    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369351    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369354    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369356    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369359    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369362    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369364    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369367    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369369    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369372    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369374    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369377    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369380    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369382    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:25:58.372450 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369385    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369387    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369390    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369392    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369394    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369397    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369400    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369402    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369406    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369408    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369411    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369414    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369417    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369419    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369422    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369426    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369962    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369968    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369972    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:25:58.372913 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369975    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369978    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369981    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369984    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369986    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369989    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369992    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369994    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369997    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.369999    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370002    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370005    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370008    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370010    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370013    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370015    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370018    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370020    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370023    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:25:58.373393 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370025    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370029    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370054    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370059    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370063    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370067    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370070    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370072    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370075    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370078    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370081    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370084    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370086    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370089    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370092    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370095    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370098    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370101    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370104    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:25:58.373853 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370106    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370109    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370111    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370114    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370116    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370118    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370121    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370124    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370126    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370129    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370132    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370134    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370137    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370140    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370142    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370145    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370148    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370151    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370154    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370157    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:25:58.374320 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370159    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370162    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370165    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370167    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370170    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370173    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370175    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370178    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370181    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370183    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370186    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370188    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370191    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370194    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370197    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370199    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370202    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370205    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370208    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:25:58.374789 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370210    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370214    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370218    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370221    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370224    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370227    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370229    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370232    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370235    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370237    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370240    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370243    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370246    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370249    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370251    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370254    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370256    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370259    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370261    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370264    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:25:58.375297 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370266    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370269    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370271    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370274    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370276    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.370279    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371245    2574 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371254    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371260    2574 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371265    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371271    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371274    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371278    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371282    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371286    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371289    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371293    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371296    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371299    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371302    2574 flags.go:64] FLAG: --cgroup-root=""
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371305    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371308    2574 flags.go:64] FLAG: --client-ca-file=""
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371311    2574 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371317    2574 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 19:25:58.375775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371322    2574 flags.go:64] FLAG: --cluster-domain=""
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371325    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371328    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371332    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371335    2574 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371340    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371343    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371346    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371349    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371352    2574 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371355    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371358    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371364    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371367    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371371    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371374    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371377    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371380    2574 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371383    2574 flags.go:64] FLAG: --enable-server="true"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371386    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371390    2574 flags.go:64] FLAG: --event-burst="100"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371393    2574 flags.go:64] FLAG: --event-qps="50"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371397    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371400    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 19:25:58.376334 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371403    2574 flags.go:64] FLAG: --eviction-hard=""
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371407    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371410    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371413    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371416    2574 flags.go:64] FLAG: --eviction-soft=""
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371419    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371422    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371429    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371432    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371435    2574 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371438    2574 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371441    2574 flags.go:64] FLAG: --feature-gates=""
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371445    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371448    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371451    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371454    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371457    2574 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371460    2574 flags.go:64] FLAG: --help="false"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371463    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371467    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371471    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371475    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371478    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 19:25:58.376924 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371481    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371484    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371487    2574 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371490    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371493    2574 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371496    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371499    2574 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371502    2574 flags.go:64] FLAG: --kube-reserved=""
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371505    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371507    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371510    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371513    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371516    2574 flags.go:64] FLAG: --lock-file=""
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371519    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371522    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371525    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371532    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371535    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371538    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371541    2574 flags.go:64] FLAG: --logging-format="text"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371544    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371547    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371551    2574 flags.go:64] FLAG: --manifest-url=""
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371554    2574 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371558    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 19:25:58.377614 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371561    2574 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371565    2574 flags.go:64] FLAG: --max-pods="110"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371568    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371571    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371575    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371578    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371580    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371583    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371586    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371594    2574 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371597    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371600    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371603    2574 flags.go:64] FLAG: --pod-cidr=""
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371606    2574 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371609    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371612    2574 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371615    2574 flags.go:64] FLAG: --port="10250"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371618    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371621    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0a7e2189fb8b0b189"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371624    2574 flags.go:64] FLAG: --qos-reserved=""
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371627    2574 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371630    2574 flags.go:64] FLAG: --register-node="true"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371633    2574 flags.go:64] FLAG: --register-with-taints=""
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371637    2574 flags.go:64] FLAG: --registry-burst="10"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371640    2574 flags.go:64] FLAG: --registry-qps="5"
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371643    2574 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 19:25:58.378708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371646    2574 flags.go:64] FLAG: --reserved-memory=""
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371650    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371653    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371656    2574 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371659    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371662    2574 flags.go:64] FLAG: --runonce="false"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371665    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371668    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371671    2574 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371673    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371677    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371679    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371682    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371686    2574 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371688    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371691    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371694    2574 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371697    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371700    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371703    2574 flags.go:64] FLAG: --system-cgroups=""
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371706    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371711    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371713    2574 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371716    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371721    2574 flags.go:64] FLAG: --tls-min-version=""
Nov 03 19:25:58.379849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371723    2574 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371726    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371729    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371732    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371735    2574 flags.go:64] FLAG: --v="2"
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371739    2574 flags.go:64] FLAG: --version="false"
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371743    2574 flags.go:64] FLAG: --vmodule=""
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371747    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.371751    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371921    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371925    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371928    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371931    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371933    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371936    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371939    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371942    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371944    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371947    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371949    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371952    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:25:58.380935 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371955    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371957    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371960    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371962    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371965    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371968    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371970    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371974    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371978    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371981    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371984    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371987    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371989    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371991    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371995    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.371997    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372000    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372003    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372005    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:25:58.381859 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372008    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372011    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372014    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372017    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372019    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372022    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372024    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372027    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372029    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372032    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372035    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372048    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372051    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372053    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372056    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372058    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372061    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372064    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372066    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372069    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:25:58.382407 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372071    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372074    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372076    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372079    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372082    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372084    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372087    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372089    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372092    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372095    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372097    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372100    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372102    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372105    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372108    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372110    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372113    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372116    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372118    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:25:58.383379 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372120    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372123    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372127    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372130    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372132    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372135    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372137    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372140    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372143    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372145    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372148    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372150    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372153    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372156    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372158    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372161    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372163    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372166    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372168    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372171    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:25:58.384074 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372173    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372177    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372180    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372182    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372185    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372188    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372190    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372193    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372198    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372202    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372205    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372207    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372210    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372213    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372217    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372220    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:25:58.384664 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.372246    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372291    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372296    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372299    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372301    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372304    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372307    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372309    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372312    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372315    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372317    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372320    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372322    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372325    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372329    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372332    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372335    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372338    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372341    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:25:58.385109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372344    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372348    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372352    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372355    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372358    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372361    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372364    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372367    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372369    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372372    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372375    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372377    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372381    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372384    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372386    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372389    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372391    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372395    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372397    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:25:58.385617 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372400    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372402    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372405    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372407    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372410    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372412    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372415    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372418    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372420    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372423    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372425    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372429    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372432    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372434    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372436    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372439    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372442    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372444    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372447    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372450    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:25:58.386098 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372452    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372456    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372458    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372461    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372464    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372468    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372471    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372474    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372477    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372479    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372482    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372484    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372487    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372489    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372492    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372495    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372497    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372500    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372502    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372505    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:25:58.386643 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372507    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372510    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372512    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372516    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372519    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372521    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372524    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372527    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372529    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372532    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372534    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372536    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372539    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372542    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372545    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372547    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372550    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372553    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372556    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372559    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:25:58.387281 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372561    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372565    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372569    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372571    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372574    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372577    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372579    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372582    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.372585    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.379753    2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.379774    2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.379802    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.379810    2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379917    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379926    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379930    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379936    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379941    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379945    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:25:58.387769 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379949    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379954    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379957    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379962    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379966    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379970    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379974    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379979    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379983    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379987    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379991    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.379996    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380000    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380004    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380009    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380014    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380018    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380022    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380027    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380032    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:25:58.388365 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380036    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380057    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380061    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380068    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380073    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380077    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380081    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380086    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380090    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380094    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380099    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380103    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380107    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380114    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380120    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380125    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380130    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380134    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380138    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380143    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:25:58.389109 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380147    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380151    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380156    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380160    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380165    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380170    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380174    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380178    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380183    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380187    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380191    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380196    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380200    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380205    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380209    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380213    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380219    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380223    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380228    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380232    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:25:58.389796 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380236    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380240    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380245    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380251    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380255    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380259    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380264    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380268    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380273    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380277    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380282    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380286    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380290    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380295    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380299    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380304    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380308    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380312    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380316    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380320    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:25:58.390290 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380324    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380329    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380333    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380337    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380341    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380346    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380350    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380354    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380359    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380365    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380369    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380374    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380378    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380385    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380393    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380398    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380404    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380408    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380413    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:25:58.390806 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380418    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.380461    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380678    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380688    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380694    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380698    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380702    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380706    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380711    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380715    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380721    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380729    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380734    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380738    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380742    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:25:58.391277 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380747    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380751    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380755    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380759    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380763    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380768    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380772    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380776    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380781    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380786    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380790    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380794    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380798    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380802    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380807    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380811    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380815    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380820    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380824    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380828    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:25:58.391634 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380832    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380837    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380841    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380845    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380849    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380854    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380858    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380862    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380866    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380870    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380875    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380879    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380884    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380888    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380892    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380897    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380901    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380906    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380910    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380914    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:25:58.392154 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380919    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380924    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380928    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380932    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380937    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380941    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380945    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380949    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380960    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380965    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380969    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380974    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380978    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380982    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380986    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380990    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380995    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.380999    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381003    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:25:58.392620 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381007    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381011    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381015    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381019    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381025    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381029    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381033    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381052    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381057    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381061    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381065    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381069    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381074    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381079    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381083    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381088    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381092    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381096    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381101    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:25:58.393089 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381106    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381110    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381114    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381118    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381122    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381127    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381131    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381138    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381143    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381147    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381151    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381156    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381160    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381164    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381169    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.381211    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 19:25:58.393530 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381265    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381272    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381277    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381282    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381286    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381291    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381295    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381299    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381304    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381308    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381312    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381317    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381321    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381326    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381330    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381334    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381339    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381343    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381349    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:25:58.393945 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381356    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381360    2574 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381366    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381370    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381375    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381380    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381385    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381389    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381393    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381397    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381402    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381406    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381410    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381414    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381418    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381425    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381430    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381435    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381439    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381446    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:25:58.394582 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381451    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381455    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381461    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381466    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381470    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381474    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381478    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381483    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381487    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381492    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381495    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381500    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381504    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381508    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381513    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381517    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381521    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381525    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381530    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381534    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:25:58.395094 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381538    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381542    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381546    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381551    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381555    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381559    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381563    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381568    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381572    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381576    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381580    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381586    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381591    2574 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381595    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381601    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381605    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381609    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381614    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381619    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381623    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:25:58.395560 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381628    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381633    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381637    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381642    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381646    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381650    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381654    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381658    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381663    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381667    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381671    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381676    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381680    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381684    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381688    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381692    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381697    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381701    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381705    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:25:58.396065 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381709    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381714    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381718    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381722    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381729    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381734    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381738    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:58.381743    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.381818    2574 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.384505    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.385499    2574 server.go:1014] "Starting client certificate rotation"
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.385595    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.385630    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.388021    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 19:25:58.396518 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.390289    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 19:25:58.399591 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.399574    2574 log.go:25] "Validated CRI v1 runtime API"
Nov 03 19:25:58.402760 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.402747    2574 log.go:25] "Validated CRI v1 image API"
Nov 03 19:25:58.403597 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.403570    2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 19:25:58.406974 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.406955    2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 95117dbc-686b-40f5-ad1d-d9492c794f91:/dev/nvme0n1p3 e2e85afc-588d-4c23-b0c8-14f5d45a6866:/dev/nvme0n1p4]
Nov 03 19:25:58.407056 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.406972    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 03 19:25:58.412737 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.412636    2574 manager.go:217] Machine: {Timestamp:2026-11-03 19:25:58.412173004 +0000 UTC m=+0.346710787 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099307 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2be45e6ee7f30ccf3893150d8d6d08 SystemUUID:ec2be45e-6ee7-f30c-cf38-93150d8d6d08 BootID:898a0088-c283-46cd-b993-4b4e8869bda5 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:dc:36:11:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:dc:36:11:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:c6:31:90:c0:03 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 19:25:58.412737 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.412735    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 19:25:58.412874 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.412803    2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 19:25:58.416342 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.416315    2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 19:25:58.416483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.416346    2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-212.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 19:25:58.416530 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.416485    2574 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 19:25:58.416530 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.416493    2574 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 19:25:58.416530 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.416515    2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 19:25:58.416530 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.416529    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 19:25:58.417418 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.417408    2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 19:25:58.421086 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.421074    2574 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 19:25:58.421199 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.421190    2574 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 19:25:58.421231 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.421224    2574 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 19:25:58.421260 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.421237    2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 19:25:58.421260 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.421248    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 19:25:58.421260 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.421257    2574 kubelet.go:405] "Adding apiserver pod source"
Nov 03 19:25:58.421337 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.421264    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 19:25:58.421587 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.421576    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 19:25:58.423547 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.423526    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 19:25:58.423809 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.423794    2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 19:25:58.428478 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.428461    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 19:25:58.429769 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.429755    2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 19:25:58.429818 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.429786    2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 19:25:58.431074 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.431060    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 19:25:58.431142 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.431078    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 19:25:58.431142 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.431084    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 19:25:58.431142 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.431090    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 19:25:58.431142 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.431095    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 19:25:58.431142 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.431102    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 19:25:58.432105 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.432096    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 19:25:58.432136 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.432105    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 19:25:58.432136 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.432113    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 19:25:58.432136 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.432119    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 19:25:58.432136 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.432128    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 19:25:58.432136 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.432136    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 19:25:58.432773 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.432755    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 19:25:58.432773 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.432770    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 19:25:58.437140 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.437125    2574 server.go:1263] "Started kubelet"
Nov 03 19:25:58.437227 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.437200    2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 19:25:58.437306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.437229    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 19:25:58.437358 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.437331    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 19:25:58.437876 ip-10-0-143-212 systemd[1]: Started Kubernetes Kubelet.
Nov 03 19:25:58.438451 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.438434    2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 19:25:58.439894 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.439879    2574 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 19:25:58.441765 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.441746    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-212.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 19:25:58.443885 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.443858    2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 03 19:25:58.448274 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.448254    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 19:25:58.449465 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.449450    2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 19:25:58.449564 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.449545    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:58.449985 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.448894    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-212.us-east-2.compute.internal.18e49f881632c459  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-212.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-212.us-east-2.compute.internal,},FirstTimestamp:2026-11-03 19:25:58.437094489 +0000 UTC m=+0.371632267,LastTimestamp:2026-11-03 19:25:58.437094489 +0000 UTC m=+0.371632267,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-212.us-east-2.compute.internal,}"
Nov 03 19:25:58.450107 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.450057    2574 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 19:25:58.450107 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.450063    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 19:25:58.450107 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.450071    2574 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 19:25:58.450231 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.450160    2574 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 19:25:58.450231 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.450171    2574 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 19:25:58.450337 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.450319    2574 factory.go:55] Registering systemd factory
Nov 03 19:25:58.450337 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.450338    2574 factory.go:223] Registration of the systemd container factory successfully
Nov 03 19:25:58.450490 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.450375    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 03 19:25:58.450490 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.450373    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-212.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 03 19:25:58.450557 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.450514    2574 factory.go:153] Registering CRI-O factory
Nov 03 19:25:58.450557 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.450527    2574 factory.go:223] Registration of the crio container factory successfully
Nov 03 19:25:58.451067 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.451052    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 19:25:58.451105 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.451079    2574 factory.go:103] Registering Raw factory
Nov 03 19:25:58.451105 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.451097    2574 manager.go:1199] Started watching for new ooms in manager
Nov 03 19:25:58.452021 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.452008    2574 manager.go:319] Starting recovery of all containers
Nov 03 19:25:58.456271 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.455710    2574 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-212.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 03 19:25:58.459268 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.459239    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 03 19:25:58.464715 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.463987    2574 manager.go:324] Recovery completed
Nov 03 19:25:58.467116 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.467093    2574 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Nov 03 19:25:58.470193 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.470178    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 19:25:58.472137 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.472113    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 19:25:58.472202 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.472157    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 19:25:58.472202 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.472170    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 19:25:58.472734 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.472718    2574 cpu_manager.go:226] "Starting" policy="none"
Nov 03 19:25:58.472734 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.472733    2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 19:25:58.472872 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.472720    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 19:25:58.472872 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.472753    2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 19:25:58.473450 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.473436    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 19:25:58.473523 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.473463    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 19:25:58.473523 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.473474    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 19:25:58.473894 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.473822    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-212.us-east-2.compute.internal.18e49f8818498339  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-212.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-212.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-212.us-east-2.compute.internal,},FirstTimestamp:2026-11-03 19:25:58.472139577 +0000 UTC m=+0.406677356,LastTimestamp:2026-11-03 19:25:58.472139577 +0000 UTC m=+0.406677356,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-212.us-east-2.compute.internal,}"
Nov 03 19:25:58.475059 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.475028    2574 policy_none.go:50] "Start"
Nov 03 19:25:58.475059 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.475060    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 19:25:58.475185 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.475078    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 19:25:58.477168 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.477151    2574 policy_none.go:44] "Start"
Nov 03 19:25:58.477579 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.477562    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7crp4"
Nov 03 19:25:58.487833 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.487754    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-212.us-east-2.compute.internal.18e49f881849e231  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-212.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-212.us-east-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-212.us-east-2.compute.internal,},FirstTimestamp:2026-11-03 19:25:58.472163889 +0000 UTC m=+0.406701667,LastTimestamp:2026-11-03 19:25:58.472163889 +0000 UTC m=+0.406701667,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-212.us-east-2.compute.internal,}"
Nov 03 19:25:58.489531 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.489513    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7crp4"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.515075    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.515814    2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.516363    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.516386    2574 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.516405    2574 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.516442    2574 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.518837    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.522552    2574 manager.go:341] "Starting Device Plugin manager"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.522584    2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.522592    2574 server.go:87] "Starting device plugin registration server"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.522824    2574 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.522836    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.522936    2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.523016    2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.523025    2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.523559    2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 19:25:58.526283 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.523602    2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:58.617036 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.616944    2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"]
Nov 03 19:25:58.617036 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.617023    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 19:25:58.618078 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.618060    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 19:25:58.618173 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.618096    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 19:25:58.618173 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.618111    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 19:25:58.620326 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.620313    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 19:25:58.620483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.620469    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.620534 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.620500    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 19:25:58.621056 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.621024    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 19:25:58.621152 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.621071    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 19:25:58.621152 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.621091    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 19:25:58.621236 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.621215    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 19:25:58.621275 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.621243    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 19:25:58.621275 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.621253    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 19:25:58.623446 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.623430    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.623519 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.623465    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 19:25:58.623519 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.623433    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 19:25:58.625759 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.625741    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 19:25:58.625848 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.625777    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 19:25:58.625848 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.625745    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 19:25:58.625848 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.625807    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 19:25:58.625848 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.625817    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 19:25:58.625848 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.625788    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 19:25:58.625848 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.625839    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.635818 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.635795    2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.635899 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.635828    2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-212.us-east-2.compute.internal\": node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:58.658369 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.658353    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found" node="ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.660389 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.660374    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found" node="ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.665268 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.665254    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:58.751216 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.751188    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.751320 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.751220    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.751320 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.751237    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/131c0f9830433abeb8e4c85b17b38230-config\") pod \"kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"131c0f9830433abeb8e4c85b17b38230\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.766296 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.766275    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:58.851771 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.851599    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.851885 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.851783    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.851885 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.851816    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/131c0f9830433abeb8e4c85b17b38230-config\") pod \"kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"131c0f9830433abeb8e4c85b17b38230\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.851885 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.851689    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.851885 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.851868    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/131c0f9830433abeb8e4c85b17b38230-config\") pod \"kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"131c0f9830433abeb8e4c85b17b38230\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.852035 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.851891    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.866714 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.866696    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:58.961422 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.961407    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.963568 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:58.963555    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:58.967660 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:58.967636    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:59.067894 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:59.067869    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:59.168397 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:59.168365    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:59.268867 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:59.268813    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:59.369496 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:59.369459    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:59.385603 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.385582    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 03 19:25:59.385715 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.385698    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 03 19:25:59.470269 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:25:59.470243    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:25:59.494825 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.494718    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 19:20:58 +0000 UTC" deadline="2028-06-28 11:18:08.381719668 +0000 UTC"
Nov 03 19:25:59.494825 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.494758    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14463h52m8.886966632s"
Nov 03 19:25:59.542862 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.542791    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 19:25:59.554506 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.554485    2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 19:25:59.612564 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:59.612537    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod131c0f9830433abeb8e4c85b17b38230.slice/crio-6ab53da3087d52e1a1b0ef48c93ecf44e1ed66caa737f073e96aae9f70307237 WatchSource:0}: Error finding container 6ab53da3087d52e1a1b0ef48c93ecf44e1ed66caa737f073e96aae9f70307237: Status 404 returned error can't find the container with id 6ab53da3087d52e1a1b0ef48c93ecf44e1ed66caa737f073e96aae9f70307237
Nov 03 19:25:59.613427 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:25:59.613410    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod702b842a8006ce3b22ac9f5c693a8e83.slice/crio-1afb98175d567260d50e10f134c7996aadc6aa99934f2fb26bf7638e2609e947 WatchSource:0}: Error finding container 1afb98175d567260d50e10f134c7996aadc6aa99934f2fb26bf7638e2609e947: Status 404 returned error can't find the container with id 1afb98175d567260d50e10f134c7996aadc6aa99934f2fb26bf7638e2609e947
Nov 03 19:25:59.617874 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.617861    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 19:25:59.649875 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.649850    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:59.664419 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.664398    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 19:25:59.665509 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.665496    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:25:59.673129 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.673109    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 19:25:59.954019 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:25:59.953939    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 19:26:00.422610 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.422569    2574 apiserver.go:52] "Watching apiserver"
Nov 03 19:26:00.436063 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.435580    2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 19:26:00.436063 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.435914    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-s9jsw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal","openshift-multus/network-metrics-daemon-jfbpg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww","openshift-network-diagnostics/network-check-target-2l9ws","openshift-network-operator/iptables-alerter-5m8zz","kube-system/konnectivity-agent-8hvrb","openshift-dns/node-resolver-rbszw","openshift-multus/multus-5q85z","openshift-multus/multus-additional-cni-plugins-jffnz","kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal","openshift-image-registry/node-ca-ss5gj","openshift-ovn-kubernetes/ovnkube-node-8rkzm","kube-system/global-pull-secret-syncer-txdf4"]
Nov 03 19:26:00.438836 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.438374    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:00.440571 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.440543    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:00.442668 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.442646    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:00.442767 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:00.442719    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:00.443283 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.443263    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 19:26:00.443669 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.443649    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-lsdlf\""
Nov 03 19:26:00.443763 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.443726    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 19:26:00.445588 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.445380    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.445815 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.445801    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 19:26:00.445884 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.445868    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lmsjd\""
Nov 03 19:26:00.445942 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.445881    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 19:26:00.445942 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.445931    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 19:26:00.448129 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.447882    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.448469 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.448453    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 03 19:26:00.450932 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.450914    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:00.452663 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.452644    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.456058 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.455135    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.457317 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.457297    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:00.460205 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460187    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.460304 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460235    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.460304 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460292    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cnibin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.460447 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460319    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-bin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.460447 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460345    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-multus\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.460447 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460370    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-daemon-config\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.460447 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460395    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-conf\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.460447 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460416    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-run\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.460447 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460441    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-var-lib-kubelet\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.460757 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460469    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ce9d2c60-8124-4576-835d-581115d6566f-iptables-alerter-script\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:00.460757 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460493    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-system-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.460757 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460515    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-k8s-cni-cncf-io\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.460757 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460604    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-registration-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.460757 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460652    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-konnectivity-ca\") pod \"konnectivity-agent-8hvrb\" (UID: \"4fd1c83c-4a87-43b1-b25e-8b9b23922aab\") " pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:00.460757 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460754    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460774    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce9d2c60-8124-4576-835d-581115d6566f-host-slash\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460788    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-socket-dir-parent\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-kubelet\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460830    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-multus-certs\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460851    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460875    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca7d9994-2fc4-4328-9c50-918954121ead-host\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460896    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-netns\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460921    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x5t9\" (UniqueName: \"kubernetes.io/projected/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kube-api-access-8x5t9\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.460960    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-tuned\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461007    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ca7d9994-2fc4-4328-9c50-918954121ead-serviceca\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461071    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m94sg\" (UniqueName: \"kubernetes.io/projected/ca7d9994-2fc4-4328-9c50-918954121ead-kube-api-access-m94sg\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:00.461111 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461103    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.461682 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461127    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-device-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.461682 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461173    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-systemd\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.461682 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-lib-modules\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.461682 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461232    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-tmp\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.461682 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461258    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-agent-certs\") pod \"konnectivity-agent-8hvrb\" (UID: \"4fd1c83c-4a87-43b1-b25e-8b9b23922aab\") " pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:00.461682 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461545    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhjvn\" (UniqueName: \"kubernetes.io/projected/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-kube-api-access-nhjvn\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:00.461951 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461793    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-socket-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.461951 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461842    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-kubernetes\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.461951 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461900    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.461951 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.461945    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlz47\" (UniqueName: \"kubernetes.io/projected/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-kube-api-access-zlz47\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.462171 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.462010    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dscb\" (UniqueName: \"kubernetes.io/projected/ce9d2c60-8124-4576-835d-581115d6566f-kube-api-access-2dscb\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:00.462171 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.462070    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cni-binary-copy\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.462389 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.462372    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-conf-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.462444 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.462401    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl2cz\" (UniqueName: \"kubernetes.io/projected/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-kube-api-access-vl2cz\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.462444 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.462420    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-host\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.462444 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.462439    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-hostroot\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.462593 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.462462    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-etc-kubernetes\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.462593 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.462481    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-sys-fs\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.462593 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.462506    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-modprobe-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.462593 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.462524    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysconfig\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.462593 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.462541    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-sys\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.464606 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.463074    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:00.464606 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:00.463205    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:00.465854 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.465835    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:00.465977 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:00.465953    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:00.467687 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.467665    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 19:26:00.479190 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.479167    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 19:26:00.481925 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.481905    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gmqw5"
Nov 03 19:26:00.498783 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.498760    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gmqw5"
Nov 03 19:26:00.523977 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.523930    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal" event={"ID":"131c0f9830433abeb8e4c85b17b38230","Type":"ContainerStarted","Data":"6ab53da3087d52e1a1b0ef48c93ecf44e1ed66caa737f073e96aae9f70307237"}
Nov 03 19:26:00.525728 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.525702    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal" event={"ID":"702b842a8006ce3b22ac9f5c693a8e83","Type":"ContainerStarted","Data":"1afb98175d567260d50e10f134c7996aadc6aa99934f2fb26bf7638e2609e947"}
Nov 03 19:26:00.543630 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.543605    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 19:26:00.543630 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.543613    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:00.543810 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.543646    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 19:26:00.543810 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.543769    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:00.543910 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.543812    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 19:26:00.543910 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.543826    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 19:26:00.543910 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.543861    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 19:26:00.543910 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.543902    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 19:26:00.544110 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.543947    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:00.544110 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.543999    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 19:26:00.544110 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.544086    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:00.544279 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.544253    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 19:26:00.544377 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.544353    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 19:26:00.544414 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.544403    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 19:26:00.546175 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.546124    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 19:26:00.546315 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.546186    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 19:26:00.546315 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.546308    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 19:26:00.546624 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.546607    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r6b5k\""
Nov 03 19:26:00.546698 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.546632    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 19:26:00.546698 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.546651    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 19:26:00.546698 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.546685    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 19:26:00.548350 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.548282    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-hwgf7\""
Nov 03 19:26:00.548350 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.548322    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 19:26:00.548856 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.548838    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xp5ps\""
Nov 03 19:26:00.548856 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.548849    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-s8pjd\""
Nov 03 19:26:00.548856 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.548856    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-st58b\""
Nov 03 19:26:00.549068 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.548858    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f8cx9\""
Nov 03 19:26:00.549068 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.548917    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g6kk8\""
Nov 03 19:26:00.549068 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.548921    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 19:26:00.551462 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.551448    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 19:26:00.563727 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563703    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-netns\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.563841 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563737    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-log-socket\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.563841 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563772    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-env-overrides\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.563841 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563799    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca7d9994-2fc4-4328-9c50-918954121ead-host\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:00.563841 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563826    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x5t9\" (UniqueName: \"kubernetes.io/projected/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kube-api-access-8x5t9\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563846    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563870    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovn-node-metrics-cert\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563890    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-binary-copy\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563904    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x86ps\" (UniqueName: \"kubernetes.io/projected/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-kube-api-access-x86ps\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ca7d9994-2fc4-4328-9c50-918954121ead-serviceca\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563943    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563956    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-lib-modules\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.563977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-system-cni-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564023    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfg6f\" (UniqueName: \"kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f\") pod \"network-check-target-2l9ws\" (UID: \"f21610fc-0c1e-4ba3-b54e-d3db441d629f\") " pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564054    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-agent-certs\") pod \"konnectivity-agent-8hvrb\" (UID: \"4fd1c83c-4a87-43b1-b25e-8b9b23922aab\") " pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564069    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhjvn\" (UniqueName: \"kubernetes.io/projected/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-kube-api-access-nhjvn\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564083    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-socket-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564097    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-kubernetes\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.564115 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564119    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlz47\" (UniqueName: \"kubernetes.io/projected/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-kube-api-access-zlz47\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564146    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjx92\" (UniqueName: \"kubernetes.io/projected/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-kube-api-access-pjx92\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cni-binary-copy\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564193    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl2cz\" (UniqueName: \"kubernetes.io/projected/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-kube-api-access-vl2cz\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564214    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-host\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564236    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84hdd\" (UniqueName: \"kubernetes.io/projected/61c81f40-4aae-4c97-9be9-30da0effd8cb-kube-api-access-84hdd\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564261    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564288    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-hostroot\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564310    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-modprobe-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564334    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-sys\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564353    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-systemd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564385    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-var-lib-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564410    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cnibin\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564431    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564445    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cnibin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564460    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-bin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564497    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-run\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.564708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564512    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-var-lib-kubelet\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564527    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-script-lib\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564543    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-system-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-registration-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564584    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61c81f40-4aae-4c97-9be9-30da0effd8cb-hosts-file\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564597    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-dbus\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564615    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce9d2c60-8124-4576-835d-581115d6566f-host-slash\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564630    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-multus-certs\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564659    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564673    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61c81f40-4aae-4c97-9be9-30da0effd8cb-tmp-dir\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564687    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-slash\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564703    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564728    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-netns\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564744    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-tuned\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564759    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-ovn\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564792    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-bin\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m94sg\" (UniqueName: \"kubernetes.io/projected/ca7d9994-2fc4-4328-9c50-918954121ead-kube-api-access-m94sg\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:00.565483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564839    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-device-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564858    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-systemd\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564884    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-tmp\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564910    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-etc-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564935    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-netd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564961    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.564986    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-config\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565011    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dscb\" (UniqueName: \"kubernetes.io/projected/ce9d2c60-8124-4576-835d-581115d6566f-kube-api-access-2dscb\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565058    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-conf-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565085    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-etc-kubernetes\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565114    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-sys-fs\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565139    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysconfig\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565159    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565178    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565214    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-multus\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565236    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-daemon-config\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.566197 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565252    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-conf\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ce9d2c60-8124-4576-835d-581115d6566f-iptables-alerter-script\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565291    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-k8s-cni-cncf-io\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565313    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-kubelet-config\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565331    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-systemd-units\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565346    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-node-log\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565362    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-konnectivity-ca\") pod \"konnectivity-agent-8hvrb\" (UID: \"4fd1c83c-4a87-43b1-b25e-8b9b23922aab\") " pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565379    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565400    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-socket-dir-parent\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565417    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-kubelet\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565441    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-kubelet\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.565603    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca7d9994-2fc4-4328-9c50-918954121ead-host\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.566282    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.566330    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ca7d9994-2fc4-4328-9c50-918954121ead-serviceca\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.566341    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.566403    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-lib-modules\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.566457    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-netns\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.566887 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.566635    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-host\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.567780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.566676    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-hostroot\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.567780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.566715    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-device-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.567780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.566750    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-systemd\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.567780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.566753    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-modprobe-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.567780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.566854    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.567780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.567053    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-conf-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.567780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.567098    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-etc-kubernetes\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.567780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.567147    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-sys-fs\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.567780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.567175    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysconfig\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.567780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.567319    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.567780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.567346    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-multus\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.570443 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.570391    2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 19:26:00.570536 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.570486    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-daemon-config\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.570656 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.570636    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-conf\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.570709    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-konnectivity-ca\") pod \"konnectivity-agent-8hvrb\" (UID: \"4fd1c83c-4a87-43b1-b25e-8b9b23922aab\") " pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.570825    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-kubelet\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:00.570872    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.570963    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-system-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.570930    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-run\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.571020    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-registration-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.571089    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cnibin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.571123    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-var-lib-kubelet\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.571138    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-socket-dir-parent\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.571172    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-bin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.571176    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ce9d2c60-8124-4576-835d-581115d6566f-iptables-alerter-script\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.571225    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-kubernetes\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.571247    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-k8s-cni-cncf-io\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.571355 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.571303    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce9d2c60-8124-4576-835d-581115d6566f-host-slash\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:00.574140 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:00.571603    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs podName:ac2071b7-e0b9-4761-ae26-bc5c0ee993d8 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:01.071555205 +0000 UTC m=+3.006092987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs") pod "network-metrics-daemon-jfbpg" (UID: "ac2071b7-e0b9-4761-ae26-bc5c0ee993d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:00.574140 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.571645    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-socket-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.574140 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.571991    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cni-binary-copy\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.574140 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.572261    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-multus-certs\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.574140 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.572322    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-sys\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.574884 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.574865    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-tmp\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.576080 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.575985    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-tuned\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.579576 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.579553    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m94sg\" (UniqueName: \"kubernetes.io/projected/ca7d9994-2fc4-4328-9c50-918954121ead-kube-api-access-m94sg\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:00.579659 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.579556    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dscb\" (UniqueName: \"kubernetes.io/projected/ce9d2c60-8124-4576-835d-581115d6566f-kube-api-access-2dscb\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:00.580132 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.580116    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl2cz\" (UniqueName: \"kubernetes.io/projected/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-kube-api-access-vl2cz\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.580345 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.580325    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x5t9\" (UniqueName: \"kubernetes.io/projected/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kube-api-access-8x5t9\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.580645 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.580627    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-agent-certs\") pod \"konnectivity-agent-8hvrb\" (UID: \"4fd1c83c-4a87-43b1-b25e-8b9b23922aab\") " pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:00.584936 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.584913    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhjvn\" (UniqueName: \"kubernetes.io/projected/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-kube-api-access-nhjvn\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:00.585022 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.584944    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlz47\" (UniqueName: \"kubernetes.io/projected/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-kube-api-access-zlz47\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.666337 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666300    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-ovn\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666337 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666350    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-bin\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666582 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666413    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-bin\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666582 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666413    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-ovn\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666582 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666462    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-etc-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666582 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666489    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-netd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666582 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666511    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-config\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666582 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666534    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-etc-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666582 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666538    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666603    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666673    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666672    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-netd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666698    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-kubelet-config\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666733    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-systemd-units\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666756    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-node-log\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666802    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-kubelet\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666826    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-netns\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666829    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-systemd-units\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666850    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-log-socket\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666855    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-node-log\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666822    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-kubelet-config\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666883    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-netns\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.666901 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666907    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-kubelet\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666872    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-env-overrides\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666957    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.666975    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovn-node-metrics-cert\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667003    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-binary-copy\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667031    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x86ps\" (UniqueName: \"kubernetes.io/projected/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-kube-api-access-x86ps\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667073    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667098    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-system-cni-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667120    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfg6f\" (UniqueName: \"kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f\") pod \"network-check-target-2l9ws\" (UID: \"f21610fc-0c1e-4ba3-b54e-d3db441d629f\") " pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667147    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjx92\" (UniqueName: \"kubernetes.io/projected/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-kube-api-access-pjx92\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667174    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84hdd\" (UniqueName: \"kubernetes.io/projected/61c81f40-4aae-4c97-9be9-30da0effd8cb-kube-api-access-84hdd\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667213    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667239    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-systemd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667262    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-var-lib-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667284    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cnibin\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667284    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.667544 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667306    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667336    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-script-lib\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667337    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-config\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667363    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61c81f40-4aae-4c97-9be9-30da0effd8cb-hosts-file\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667383    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-env-overrides\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:00.667440    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667445    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-var-lib-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:00.667497    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret podName:074a9efa-d51e-4192-841c-3cd308d89945 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:01.167481233 +0000 UTC m=+3.102018999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret") pod "global-pull-secret-syncer-txdf4" (UID: "074a9efa-d51e-4192-841c-3cd308d89945") : object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667495    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-systemd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667530    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-system-cni-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667522    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cnibin\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667697    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667747    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667775    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-dbus\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667797    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61c81f40-4aae-4c97-9be9-30da0effd8cb-hosts-file\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667817    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61c81f40-4aae-4c97-9be9-30da0effd8cb-tmp-dir\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-slash\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668306 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667866    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.668786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667939    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-dbus\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:00.668786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667869    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.667975    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-binary-copy\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.668786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.668009    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-script-lib\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.668027    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-slash\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.668030    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.668153    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-log-socket\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.668786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.668305    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61c81f40-4aae-4c97-9be9-30da0effd8cb-tmp-dir\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:00.670273 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.670248    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovn-node-metrics-cert\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.684209 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:00.684147    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 19:26:00.684209 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:00.684175    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 19:26:00.684209 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:00.684187    2574 projected.go:196] Error preparing data for projected volume kube-api-access-zfg6f for pod openshift-network-diagnostics/network-check-target-2l9ws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:00.684425 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:00.684263    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f podName:f21610fc-0c1e-4ba3-b54e-d3db441d629f nodeName:}" failed. No retries permitted until 2026-11-03 19:26:01.18424652 +0000 UTC m=+3.118784289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zfg6f" (UniqueName: "kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f") pod "network-check-target-2l9ws" (UID: "f21610fc-0c1e-4ba3-b54e-d3db441d629f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:00.691061 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.686571    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjx92\" (UniqueName: \"kubernetes.io/projected/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-kube-api-access-pjx92\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.691061 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.687423    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84hdd\" (UniqueName: \"kubernetes.io/projected/61c81f40-4aae-4c97-9be9-30da0effd8cb-kube-api-access-84hdd\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:00.691061 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.690920    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x86ps\" (UniqueName: \"kubernetes.io/projected/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-kube-api-access-x86ps\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.753770 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.753738    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:00.763054 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.763014    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:00.764576 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:00.764548    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd1c83c_4a87_43b1_b25e_8b9b23922aab.slice/crio-89d7980e22f6af81ccdd1c7a97930400d2296be499e569c25b9a1ba7f618fe5e WatchSource:0}: Error finding container 89d7980e22f6af81ccdd1c7a97930400d2296be499e569c25b9a1ba7f618fe5e: Status 404 returned error can't find the container with id 89d7980e22f6af81ccdd1c7a97930400d2296be499e569c25b9a1ba7f618fe5e
Nov 03 19:26:00.772369 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:00.772337    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca7d9994_2fc4_4328_9c50_918954121ead.slice/crio-d6aebe412b486b3379cd24a3b80fa9be586a5f39e14a501de9b49ae18378f436 WatchSource:0}: Error finding container d6aebe412b486b3379cd24a3b80fa9be586a5f39e14a501de9b49ae18378f436: Status 404 returned error can't find the container with id d6aebe412b486b3379cd24a3b80fa9be586a5f39e14a501de9b49ae18378f436
Nov 03 19:26:00.773026 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.773006    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5q85z"
Nov 03 19:26:00.782748 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:00.782717    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfff1c16_19ba_4492_aacb_7b75bdd64c9c.slice/crio-5e15edcc9f9b1866c9b9b56e137ee21d731d367d50f40ac6bc9fbc03889771ae WatchSource:0}: Error finding container 5e15edcc9f9b1866c9b9b56e137ee21d731d367d50f40ac6bc9fbc03889771ae: Status 404 returned error can't find the container with id 5e15edcc9f9b1866c9b9b56e137ee21d731d367d50f40ac6bc9fbc03889771ae
Nov 03 19:26:00.784651 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.784630    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:00.787548 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.787528    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:00.796535 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.796384    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:00.797416 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:00.797390    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4236bfc_5fb6_40aa_8ced_a63a86f81632.slice/crio-c7d0befb2d6c96253f3d2eb0ce3b61b33be7c9f578898ffa084b3e50902ecfa0 WatchSource:0}: Error finding container c7d0befb2d6c96253f3d2eb0ce3b61b33be7c9f578898ffa084b3e50902ecfa0: Status 404 returned error can't find the container with id c7d0befb2d6c96253f3d2eb0ce3b61b33be7c9f578898ffa084b3e50902ecfa0
Nov 03 19:26:00.798370 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:00.798159    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97ebe2a8_5c4e_44f5_bbbe_46e49f5f55a0.slice/crio-ce690c8c07dd4ec0d1e8223df602cebf887d326889b76a427612012adada68a8 WatchSource:0}: Error finding container ce690c8c07dd4ec0d1e8223df602cebf887d326889b76a427612012adada68a8: Status 404 returned error can't find the container with id ce690c8c07dd4ec0d1e8223df602cebf887d326889b76a427612012adada68a8
Nov 03 19:26:00.805993 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:00.805970    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce9d2c60_8124_4576_835d_581115d6566f.slice/crio-5e36bf1426511d0dba9ca46de226dbcdfab94f5468443da6ba9e8d0ccfe5ede4 WatchSource:0}: Error finding container 5e36bf1426511d0dba9ca46de226dbcdfab94f5468443da6ba9e8d0ccfe5ede4: Status 404 returned error can't find the container with id 5e36bf1426511d0dba9ca46de226dbcdfab94f5468443da6ba9e8d0ccfe5ede4
Nov 03 19:26:00.816531 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.816506    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:00.825137 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.825116    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:00.826707 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:00.826667    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c81f40_4aae_4c97_9be9_30da0effd8cb.slice/crio-ac6167d831859048646891f758c7383cfe54256ff8147e6c39e133a6bdd5c62a WatchSource:0}: Error finding container ac6167d831859048646891f758c7383cfe54256ff8147e6c39e133a6bdd5c62a: Status 404 returned error can't find the container with id ac6167d831859048646891f758c7383cfe54256ff8147e6c39e133a6bdd5c62a
Nov 03 19:26:00.833946 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:00.833925    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180203ac_0a6f_4554_8ac7_aca9b55c7b3c.slice/crio-19855996f8e578426a6a412f77cba7918b2ca057e717cdd919be47382e40f72f WatchSource:0}: Error finding container 19855996f8e578426a6a412f77cba7918b2ca057e717cdd919be47382e40f72f: Status 404 returned error can't find the container with id 19855996f8e578426a6a412f77cba7918b2ca057e717cdd919be47382e40f72f
Nov 03 19:26:00.834946 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:00.834924    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:00.845091 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:00.845069    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a18e52_fbcd_4f04_a5f5_2ebafaea516a.slice/crio-c2bf2d36aec738909756a0ad609e9e57065d5b1eebfcfeba17a7ea5c518fa206 WatchSource:0}: Error finding container c2bf2d36aec738909756a0ad609e9e57065d5b1eebfcfeba17a7ea5c518fa206: Status 404 returned error can't find the container with id c2bf2d36aec738909756a0ad609e9e57065d5b1eebfcfeba17a7ea5c518fa206
Nov 03 19:26:01.171080 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.171030    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:01.171245 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.171099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:01.171662 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:01.171636    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:01.171793 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:01.171713    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs podName:ac2071b7-e0b9-4761-ae26-bc5c0ee993d8 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:02.171693303 +0000 UTC m=+4.106231070 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs") pod "network-metrics-daemon-jfbpg" (UID: "ac2071b7-e0b9-4761-ae26-bc5c0ee993d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:01.172155 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:01.172067    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:01.172155 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:01.172133    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret podName:074a9efa-d51e-4192-841c-3cd308d89945 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:02.172116058 +0000 UTC m=+4.106653828 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret") pod "global-pull-secret-syncer-txdf4" (UID: "074a9efa-d51e-4192-841c-3cd308d89945") : object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:01.271650 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.271615    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfg6f\" (UniqueName: \"kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f\") pod \"network-check-target-2l9ws\" (UID: \"f21610fc-0c1e-4ba3-b54e-d3db441d629f\") " pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:01.272254 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:01.271800    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 19:26:01.272254 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:01.271828    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 19:26:01.272254 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:01.271843    2574 projected.go:196] Error preparing data for projected volume kube-api-access-zfg6f for pod openshift-network-diagnostics/network-check-target-2l9ws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:01.272254 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:01.271903    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f podName:f21610fc-0c1e-4ba3-b54e-d3db441d629f nodeName:}" failed. No retries permitted until 2026-11-03 19:26:02.271884302 +0000 UTC m=+4.206422069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zfg6f" (UniqueName: "kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f") pod "network-check-target-2l9ws" (UID: "f21610fc-0c1e-4ba3-b54e-d3db441d629f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:01.502980 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.502703    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 19:21:00 +0000 UTC" deadline="2028-05-30 22:17:09.009905392 +0000 UTC"
Nov 03 19:26:01.502980 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.502982    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13778h51m7.506930558s"
Nov 03 19:26:01.539707 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.539576    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5q85z" event={"ID":"cfff1c16-19ba-4492-aacb-7b75bdd64c9c","Type":"ContainerStarted","Data":"5e15edcc9f9b1866c9b9b56e137ee21d731d367d50f40ac6bc9fbc03889771ae"}
Nov 03 19:26:01.561169 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.561129    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hvrb" event={"ID":"4fd1c83c-4a87-43b1-b25e-8b9b23922aab","Type":"ContainerStarted","Data":"89d7980e22f6af81ccdd1c7a97930400d2296be499e569c25b9a1ba7f618fe5e"}
Nov 03 19:26:01.566703 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.566530    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rbszw" event={"ID":"61c81f40-4aae-4c97-9be9-30da0effd8cb","Type":"ContainerStarted","Data":"ac6167d831859048646891f758c7383cfe54256ff8147e6c39e133a6bdd5c62a"}
Nov 03 19:26:01.570704 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.570641    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s9jsw" event={"ID":"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0","Type":"ContainerStarted","Data":"ce690c8c07dd4ec0d1e8223df602cebf887d326889b76a427612012adada68a8"}
Nov 03 19:26:01.574141 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.573956    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww" event={"ID":"c4236bfc-5fb6-40aa-8ced-a63a86f81632","Type":"ContainerStarted","Data":"c7d0befb2d6c96253f3d2eb0ce3b61b33be7c9f578898ffa084b3e50902ecfa0"}
Nov 03 19:26:01.585788 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.585670    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ss5gj" event={"ID":"ca7d9994-2fc4-4328-9c50-918954121ead","Type":"ContainerStarted","Data":"d6aebe412b486b3379cd24a3b80fa9be586a5f39e14a501de9b49ae18378f436"}
Nov 03 19:26:01.594316 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.594251    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal" event={"ID":"131c0f9830433abeb8e4c85b17b38230","Type":"ContainerStarted","Data":"8957fda0f81e84b1289a258b4f00fdf84464f0f06a8eeadc38332b3281166f6f"}
Nov 03 19:26:01.609770 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.609610    2574 generic.go:356] "Generic (PLEG): container finished" podID="702b842a8006ce3b22ac9f5c693a8e83" containerID="f9444d5d8dfd60f96a7f19fa1ddbdbd1d1c405f76a83ba53558d3252fa3096bf" exitCode=0
Nov 03 19:26:01.609770 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.609686    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal" event={"ID":"702b842a8006ce3b22ac9f5c693a8e83","Type":"ContainerDied","Data":"f9444d5d8dfd60f96a7f19fa1ddbdbd1d1c405f76a83ba53558d3252fa3096bf"}
Nov 03 19:26:01.615036 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.614974    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jffnz" event={"ID":"40a18e52-fbcd-4f04-a5f5-2ebafaea516a","Type":"ContainerStarted","Data":"c2bf2d36aec738909756a0ad609e9e57065d5b1eebfcfeba17a7ea5c518fa206"}
Nov 03 19:26:01.619410 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.619113    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm" event={"ID":"180203ac-0a6f-4554-8ac7-aca9b55c7b3c","Type":"ContainerStarted","Data":"19855996f8e578426a6a412f77cba7918b2ca057e717cdd919be47382e40f72f"}
Nov 03 19:26:01.633328 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.633304    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5m8zz" event={"ID":"ce9d2c60-8124-4576-835d-581115d6566f","Type":"ContainerStarted","Data":"5e36bf1426511d0dba9ca46de226dbcdfab94f5468443da6ba9e8d0ccfe5ede4"}
Nov 03 19:26:01.642074 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:01.642008    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal" podStartSLOduration=2.641992653 podStartE2EDuration="2.641992653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 19:25:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:01.619104084 +0000 UTC m=+3.553641873" watchObservedRunningTime="2026-11-03 19:26:01.641992653 +0000 UTC m=+3.576530440"
Nov 03 19:26:02.181871 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:02.181789    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:02.181871 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:02.181842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:02.182155 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:02.181979    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:02.182155 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:02.182053    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret podName:074a9efa-d51e-4192-841c-3cd308d89945 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:04.182021674 +0000 UTC m=+6.116559461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret") pod "global-pull-secret-syncer-txdf4" (UID: "074a9efa-d51e-4192-841c-3cd308d89945") : object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:02.182456 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:02.182439    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:02.182521 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:02.182493    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs podName:ac2071b7-e0b9-4761-ae26-bc5c0ee993d8 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:04.182476741 +0000 UTC m=+6.117014520 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs") pod "network-metrics-daemon-jfbpg" (UID: "ac2071b7-e0b9-4761-ae26-bc5c0ee993d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:02.282884 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:02.282840    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfg6f\" (UniqueName: \"kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f\") pod \"network-check-target-2l9ws\" (UID: \"f21610fc-0c1e-4ba3-b54e-d3db441d629f\") " pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:02.283073 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:02.283028    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 19:26:02.283073 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:02.283064    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 19:26:02.283200 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:02.283077    2574 projected.go:196] Error preparing data for projected volume kube-api-access-zfg6f for pod openshift-network-diagnostics/network-check-target-2l9ws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:02.283200 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:02.283136    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f podName:f21610fc-0c1e-4ba3-b54e-d3db441d629f nodeName:}" failed. No retries permitted until 2026-11-03 19:26:04.283117746 +0000 UTC m=+6.217655531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zfg6f" (UniqueName: "kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f") pod "network-check-target-2l9ws" (UID: "f21610fc-0c1e-4ba3-b54e-d3db441d629f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:02.504079 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:02.503998    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 19:21:00 +0000 UTC" deadline="2028-05-13 01:06:57.820428659 +0000 UTC"
Nov 03 19:26:02.504079 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:02.504035    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13349h40m55.316397842s"
Nov 03 19:26:02.518093 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:02.517390    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:02.518093 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:02.517539    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:02.518093 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:02.517945    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:02.518093 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:02.518032    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:02.518626 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:02.518489    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:02.518626 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:02.518593    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:02.647195 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:02.647127    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal" event={"ID":"702b842a8006ce3b22ac9f5c693a8e83","Type":"ContainerStarted","Data":"f72a7f67bca55c61dca52964e03a32a2fb025e7ba1110ef02ce3cec64e8771a4"}
Nov 03 19:26:02.666505 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:02.666454    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal" podStartSLOduration=3.666435602 podStartE2EDuration="3.666435602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 19:25:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:02.665580077 +0000 UTC m=+4.600117866" watchObservedRunningTime="2026-11-03 19:26:02.666435602 +0000 UTC m=+4.600973391"
Nov 03 19:26:04.199275 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:04.199234    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:04.199811 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:04.199351    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:04.199811 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:04.199419    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:04.199811 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:04.199466    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:04.199811 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:04.199506    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret podName:074a9efa-d51e-4192-841c-3cd308d89945 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:08.199487046 +0000 UTC m=+10.134024813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret") pod "global-pull-secret-syncer-txdf4" (UID: "074a9efa-d51e-4192-841c-3cd308d89945") : object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:04.199811 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:04.199528    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs podName:ac2071b7-e0b9-4761-ae26-bc5c0ee993d8 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:08.1995185 +0000 UTC m=+10.134056271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs") pod "network-metrics-daemon-jfbpg" (UID: "ac2071b7-e0b9-4761-ae26-bc5c0ee993d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:04.300426 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:04.300390    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfg6f\" (UniqueName: \"kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f\") pod \"network-check-target-2l9ws\" (UID: \"f21610fc-0c1e-4ba3-b54e-d3db441d629f\") " pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:04.300595 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:04.300578    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 19:26:04.300662 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:04.300607    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 19:26:04.300662 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:04.300620    2574 projected.go:196] Error preparing data for projected volume kube-api-access-zfg6f for pod openshift-network-diagnostics/network-check-target-2l9ws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:04.300760 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:04.300678    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f podName:f21610fc-0c1e-4ba3-b54e-d3db441d629f nodeName:}" failed. No retries permitted until 2026-11-03 19:26:08.300659767 +0000 UTC m=+10.235197550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zfg6f" (UniqueName: "kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f") pod "network-check-target-2l9ws" (UID: "f21610fc-0c1e-4ba3-b54e-d3db441d629f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:04.518959 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:04.518926    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:04.519158 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:04.519080    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:04.519742 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:04.519462    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:04.519742 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:04.519555    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:04.519742 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:04.519611    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:04.519742 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:04.519692    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:04.659774 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:04.659733    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hvrb" event={"ID":"4fd1c83c-4a87-43b1-b25e-8b9b23922aab","Type":"ContainerStarted","Data":"f792ee1a7db51540ed0e405df1b63167682f30bd7bbd2e4cc761c600d50c640a"}
Nov 03 19:26:04.662746 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:04.662714    2574 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="4ba76dd23f7cb46ec2956118b65431abf0b569327517b8831dffb7d1d0a0800f" exitCode=0
Nov 03 19:26:04.662746 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:04.662761    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jffnz" event={"ID":"40a18e52-fbcd-4f04-a5f5-2ebafaea516a","Type":"ContainerDied","Data":"4ba76dd23f7cb46ec2956118b65431abf0b569327517b8831dffb7d1d0a0800f"}
Nov 03 19:26:04.711388 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:04.711306    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8hvrb" podStartSLOduration=3.212301032 podStartE2EDuration="6.711289303s" totalImagesPullingTime="3.498988271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:04.682304467 +0000 UTC m=+6.616842258" watchObservedRunningTime="2026-11-03 19:26:04.711289303 +0000 UTC m=+6.645827092"
Nov 03 19:26:05.667540 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:05.667503    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ss5gj" event={"ID":"ca7d9994-2fc4-4328-9c50-918954121ead","Type":"ContainerStarted","Data":"0089b1e360512357e52eb033657ef3250ee9b96b9407611e2b7a366f0d98c25a"}
Nov 03 19:26:06.519102 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:06.518156    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:06.519102 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:06.518292    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:06.519102 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:06.518913    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:06.519102 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:06.519009    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:06.519715 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:06.519537    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:06.519715 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:06.519640    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:06.674673 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:06.674556    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww" event={"ID":"c4236bfc-5fb6-40aa-8ced-a63a86f81632","Type":"ContainerStarted","Data":"3cfca333f3f1ce356552cd07bdb62583083cb4ffa1b4483ce3e257168ebb547d"}
Nov 03 19:26:07.440714 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:07.440653    2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 19:26:07.528496 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:07.528404    2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T19:26:07.44067294Z","UUID":"584349d2-14d5-44d4-85c0-6f3fe2714923","Handler":null,"Name":"","Endpoint":""}
Nov 03 19:26:07.529869 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:07.529847    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 19:26:07.529977 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:07.529879    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 19:26:07.683010 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:07.682973    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww" event={"ID":"c4236bfc-5fb6-40aa-8ced-a63a86f81632","Type":"ContainerStarted","Data":"99886cad4b2e481adb49031534144ef45038c558ce4cdb736d26b114add458c5"}
Nov 03 19:26:07.945684 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:07.945614    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:07.946712 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:07.946690    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:07.973674 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:07.973618    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ss5gj" podStartSLOduration=5.639261672 podStartE2EDuration="9.97360076s" totalImagesPullingTime="4.334339088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:05.686898382 +0000 UTC m=+7.621436173" watchObservedRunningTime="2026-11-03 19:26:07.97360076 +0000 UTC m=+9.908138549"
Nov 03 19:26:08.232330 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.232297    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:08.232474 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.232352    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:08.232474 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:08.232452    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:08.232547 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:08.232524    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret podName:074a9efa-d51e-4192-841c-3cd308d89945 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:16.232508056 +0000 UTC m=+18.167045826 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret") pod "global-pull-secret-syncer-txdf4" (UID: "074a9efa-d51e-4192-841c-3cd308d89945") : object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:08.232604 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:08.232452    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:08.232652 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:08.232635    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs podName:ac2071b7-e0b9-4761-ae26-bc5c0ee993d8 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:16.232619946 +0000 UTC m=+18.167157727 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs") pod "network-metrics-daemon-jfbpg" (UID: "ac2071b7-e0b9-4761-ae26-bc5c0ee993d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:08.336522 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.336495    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfg6f\" (UniqueName: \"kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f\") pod \"network-check-target-2l9ws\" (UID: \"f21610fc-0c1e-4ba3-b54e-d3db441d629f\") " pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:08.336649 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:08.336638    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 19:26:08.336705 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:08.336659    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 19:26:08.336705 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:08.336671    2574 projected.go:196] Error preparing data for projected volume kube-api-access-zfg6f for pod openshift-network-diagnostics/network-check-target-2l9ws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:08.336791 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:08.336731    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f podName:f21610fc-0c1e-4ba3-b54e-d3db441d629f nodeName:}" failed. No retries permitted until 2026-11-03 19:26:16.336713016 +0000 UTC m=+18.271250795 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zfg6f" (UniqueName: "kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f") pod "network-check-target-2l9ws" (UID: "f21610fc-0c1e-4ba3-b54e-d3db441d629f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:08.517974 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.517895    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:08.518141 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:08.518073    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:08.518443 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.518422    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:08.518564 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:08.518529    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:08.518622 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.518612    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:08.518745 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:08.518720    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:08.687758 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.687693    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jffnz" event={"ID":"40a18e52-fbcd-4f04-a5f5-2ebafaea516a","Type":"ContainerStarted","Data":"bfcf0b75159b1caec7332a0491fe6b2091030404c3ed935ae9ff0dd0a6f7d06e"}
Nov 03 19:26:08.691486 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.691455    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5q85z" event={"ID":"cfff1c16-19ba-4492-aacb-7b75bdd64c9c","Type":"ContainerStarted","Data":"a5954333b3a9751b6eebda73489029fe5be2f354de81014c6a2be73ba2847b22"}
Nov 03 19:26:08.693716 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.693685    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rbszw" event={"ID":"61c81f40-4aae-4c97-9be9-30da0effd8cb","Type":"ContainerStarted","Data":"e653dc1bbe45a2307c8b8f8e1efb44334ba7b98afd220edb7ee61b76627d71ae"}
Nov 03 19:26:08.697261 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.697241    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww" event={"ID":"c4236bfc-5fb6-40aa-8ced-a63a86f81632","Type":"ContainerStarted","Data":"34a9066f64659531ab6abbcca76a563c9aa8f11d103f56f1875b29a588d7fcd2"}
Nov 03 19:26:08.759656 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.759609    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww" podStartSLOduration=3.346794192 podStartE2EDuration="10.759595363s" totalImagesPullingTime="7.412801171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 19:25:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:08.758972517 +0000 UTC m=+10.693510300" watchObservedRunningTime="2026-11-03 19:26:08.759595363 +0000 UTC m=+10.694133149"
Nov 03 19:26:08.759861 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.759833    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-rbszw" podStartSLOduration=3.4338612 podStartE2EDuration="10.759826529s" totalImagesPullingTime="7.325965329s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:08.734741029 +0000 UTC m=+10.669278814" watchObservedRunningTime="2026-11-03 19:26:08.759826529 +0000 UTC m=+10.694364320"
Nov 03 19:26:08.780302 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:08.780224    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-5q85z" podStartSLOduration=3.931089881 podStartE2EDuration="10.780213067s" totalImagesPullingTime="6.849123186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:08.780115949 +0000 UTC m=+10.714653738" watchObservedRunningTime="2026-11-03 19:26:08.780213067 +0000 UTC m=+10.714750855"
Nov 03 19:26:09.701549 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:09.701517    2574 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="bfcf0b75159b1caec7332a0491fe6b2091030404c3ed935ae9ff0dd0a6f7d06e" exitCode=0
Nov 03 19:26:09.701930 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:09.701589    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jffnz" event={"ID":"40a18e52-fbcd-4f04-a5f5-2ebafaea516a","Type":"ContainerDied","Data":"bfcf0b75159b1caec7332a0491fe6b2091030404c3ed935ae9ff0dd0a6f7d06e"}
Nov 03 19:26:09.705341 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:09.705315    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5m8zz" event={"ID":"ce9d2c60-8124-4576-835d-581115d6566f","Type":"ContainerStarted","Data":"ed5280492edaebf85e8d17212d59157ad1dc386adf721485f17b4205b8146c7a"}
Nov 03 19:26:09.782289 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:09.782244    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5m8zz" podStartSLOduration=4.608178609 podStartE2EDuration="11.782230814s" totalImagesPullingTime="7.174052205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:09.78193988 +0000 UTC m=+11.716477666" watchObservedRunningTime="2026-11-03 19:26:09.782230814 +0000 UTC m=+11.716768600"
Nov 03 19:26:10.517305 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:10.517270    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:10.517482 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:10.517315    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:10.517482 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:10.517274    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:10.517482 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:10.517414    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:10.517633 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:10.517482    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:10.517633 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:10.517550    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:11.711975 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:11.711936    2574 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="3941407d069cdfd1259b1255d65ff50340b0305414735323755585aaa523a4b6" exitCode=0
Nov 03 19:26:11.712636 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:11.711998    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jffnz" event={"ID":"40a18e52-fbcd-4f04-a5f5-2ebafaea516a","Type":"ContainerDied","Data":"3941407d069cdfd1259b1255d65ff50340b0305414735323755585aaa523a4b6"}
Nov 03 19:26:12.516868 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:12.516678    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:12.517028 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:12.516957    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:12.517028 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:12.516812    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:12.517159 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:12.517056    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:12.517159 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:12.516804    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:12.517159 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:12.517146    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:12.717002 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:12.716971    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s9jsw" event={"ID":"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0","Type":"ContainerStarted","Data":"ea0fb31088965026b09972b9d763f512b69642b876827310c4fd66d4dcd26ce8"}
Nov 03 19:26:12.738955 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:12.738907    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-s9jsw" podStartSLOduration=3.7533051200000003 podStartE2EDuration="14.738891062s" totalImagesPullingTime="10.985585942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:12.738633667 +0000 UTC m=+14.673171515" watchObservedRunningTime="2026-11-03 19:26:12.738891062 +0000 UTC m=+14.673428851"
Nov 03 19:26:13.722113 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:13.722080    2574 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="bf05e7f6da51439621c6bc6386c2b53b7409b361b60697e758ae009cd7664d78" exitCode=0
Nov 03 19:26:13.722632 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:13.722150    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jffnz" event={"ID":"40a18e52-fbcd-4f04-a5f5-2ebafaea516a","Type":"ContainerDied","Data":"bf05e7f6da51439621c6bc6386c2b53b7409b361b60697e758ae009cd7664d78"}
Nov 03 19:26:14.516839 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:14.516797    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:14.516839 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:14.516815    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:14.517087 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:14.516970    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:14.517087 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:14.517015    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:14.517195 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:14.517111    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:14.517195 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:14.517181    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:15.729286 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:15.729103    2574 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="a32e17c370667bb5f15944502e3bced1aa740e2e56d60ac72705f0bc13c1c272" exitCode=0
Nov 03 19:26:15.729896 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:15.729171    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jffnz" event={"ID":"40a18e52-fbcd-4f04-a5f5-2ebafaea516a","Type":"ContainerDied","Data":"a32e17c370667bb5f15944502e3bced1aa740e2e56d60ac72705f0bc13c1c272"}
Nov 03 19:26:15.732517 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:15.732498    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm" event={"ID":"180203ac-0a6f-4554-8ac7-aca9b55c7b3c","Type":"ContainerStarted","Data":"198da5d6ec6fd34fa46240a5f7732c0e3ea28732646ca1bf6f71aef653c07180"}
Nov 03 19:26:15.732610 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:15.732583    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm" event={"ID":"180203ac-0a6f-4554-8ac7-aca9b55c7b3c","Type":"ContainerStarted","Data":"f209013b56e2100f5d5fc10924d798b6103967c2adb52c0783d13bbcd8e5cda8"}
Nov 03 19:26:15.732647 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:15.732614    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm" event={"ID":"180203ac-0a6f-4554-8ac7-aca9b55c7b3c","Type":"ContainerStarted","Data":"be243676d45e4e88f6075307ba620b3e224428c109a0aab69f9a4c202ea0652d"}
Nov 03 19:26:15.732647 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:15.732641    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm" event={"ID":"180203ac-0a6f-4554-8ac7-aca9b55c7b3c","Type":"ContainerStarted","Data":"33a51eafa376c83dc019ef2cb262e25a2d9c47a7c2862b6fc16827caeb83d251"}
Nov 03 19:26:15.732715 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:15.732668    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm" event={"ID":"180203ac-0a6f-4554-8ac7-aca9b55c7b3c","Type":"ContainerStarted","Data":"f4fa405d69755bfea577d966000ceadd2ed97fc82814ae95634fd551d681679e"}
Nov 03 19:26:15.732715 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:15.732694    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm" event={"ID":"180203ac-0a6f-4554-8ac7-aca9b55c7b3c","Type":"ContainerStarted","Data":"a45dd5a59a74a9f79645d55ef6fb826f514ef014f46a54fcae9c7c51d6f31bb2"}
Nov 03 19:26:16.297473 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:16.297444    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:16.297625 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:16.297481    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:16.297625 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:16.297578    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:16.297625 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:16.297579    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:16.297713 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:16.297626    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs podName:ac2071b7-e0b9-4761-ae26-bc5c0ee993d8 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:32.297612531 +0000 UTC m=+34.232150314 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs") pod "network-metrics-daemon-jfbpg" (UID: "ac2071b7-e0b9-4761-ae26-bc5c0ee993d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 19:26:16.297713 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:16.297639    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret podName:074a9efa-d51e-4192-841c-3cd308d89945 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:32.297633502 +0000 UTC m=+34.232171267 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret") pod "global-pull-secret-syncer-txdf4" (UID: "074a9efa-d51e-4192-841c-3cd308d89945") : object "kube-system"/"original-pull-secret" not registered
Nov 03 19:26:16.398175 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:16.398149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfg6f\" (UniqueName: \"kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f\") pod \"network-check-target-2l9ws\" (UID: \"f21610fc-0c1e-4ba3-b54e-d3db441d629f\") " pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:16.398322 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:16.398296    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 19:26:16.398322 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:16.398311    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 19:26:16.398322 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:16.398319    2574 projected.go:196] Error preparing data for projected volume kube-api-access-zfg6f for pod openshift-network-diagnostics/network-check-target-2l9ws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:16.398406 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:16.398360    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f podName:f21610fc-0c1e-4ba3-b54e-d3db441d629f nodeName:}" failed. No retries permitted until 2026-11-03 19:26:32.398348479 +0000 UTC m=+34.332886244 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zfg6f" (UniqueName: "kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f") pod "network-check-target-2l9ws" (UID: "f21610fc-0c1e-4ba3-b54e-d3db441d629f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 19:26:16.516874 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:16.516842    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:16.517008 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:16.516941    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:16.517008 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:16.516975    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:16.517115 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:16.516942    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:16.517115 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:16.517067    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:16.517196 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:16.517144    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:16.736911 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:16.736882    2574 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="1a8551d3ab449de72ef6368ec80c20fa99b09fa061143bdeed143966edbd596e" exitCode=0
Nov 03 19:26:16.737279 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:16.736945    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jffnz" event={"ID":"40a18e52-fbcd-4f04-a5f5-2ebafaea516a","Type":"ContainerDied","Data":"1a8551d3ab449de72ef6368ec80c20fa99b09fa061143bdeed143966edbd596e"}
Nov 03 19:26:17.742620 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:17.742586    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jffnz" event={"ID":"40a18e52-fbcd-4f04-a5f5-2ebafaea516a","Type":"ContainerStarted","Data":"e9897c8b1d651ea0fe62efa3c022075859e04556f92c5e1410471dfcb14d8c64"}
Nov 03 19:26:17.745669 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:17.745645    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm" event={"ID":"180203ac-0a6f-4554-8ac7-aca9b55c7b3c","Type":"ContainerStarted","Data":"ca2f7b5db9870680e255a0483dcb7247a6b616ba19bd29900f8e3f594afee7ea"}
Nov 03 19:26:17.773753 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:17.773711    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jffnz" podStartSLOduration=9.660484201 podStartE2EDuration="19.773699263s" totalImagesPullingTime="10.113215062s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 19:25:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:17.773270839 +0000 UTC m=+19.707808626" watchObservedRunningTime="2026-11-03 19:26:17.773699263 +0000 UTC m=+19.708237049"
Nov 03 19:26:18.172546 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:18.172476    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:18.172684 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:18.172604    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 19:26:18.172995 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:18.172977    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:18.519224 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:18.519196    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:18.519581 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:18.519552    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:18.520102 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:18.520080    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:18.520329 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:18.520309    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:18.520470 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:18.520457    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:18.520639 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:18.520624    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:19.755472 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:19.755439    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm" event={"ID":"180203ac-0a6f-4554-8ac7-aca9b55c7b3c","Type":"ContainerStarted","Data":"2db47348fc5c21249121b93cee7030be3c1c7453b7aa2bc34a616877c0b80891"}
Nov 03 19:26:19.755923 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:19.755767    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:19.755923 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:19.755831    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:19.771063 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:19.771029    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:19.795432 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:19.795394    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm" podStartSLOduration=7.786454726 podStartE2EDuration="21.795383226s" totalImagesPullingTime="14.0089285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:19.794938332 +0000 UTC m=+21.729476119" watchObservedRunningTime="2026-11-03 19:26:19.795383226 +0000 UTC m=+21.729921013"
Nov 03 19:26:20.517685 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:20.517653    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:20.517833 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:20.517718    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:20.517833 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:20.517719    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:20.517928 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:20.517825    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:20.517979 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:20.517914    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:20.518034 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:20.517989    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:20.758263 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:20.758235    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:20.772176 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:20.772116    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:20.935576 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:20.935548    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txdf4"]
Nov 03 19:26:20.935711 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:20.935646    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:20.935782 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:20.935756    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:20.936322 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:20.936300    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2l9ws"]
Nov 03 19:26:20.936386 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:20.936380    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:20.936457 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:20.936442    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:20.937535 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:20.937517    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jfbpg"]
Nov 03 19:26:20.937609 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:20.937598    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:20.937716 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:20.937697    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:22.516877 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:22.516851    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:22.516877 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:22.516868    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:22.517373 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:22.516968    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:22.517373 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:22.517013    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:22.517373 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:22.517060    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:22.517373 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:22.517163    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:24.517350 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:24.517315    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:24.517350 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:24.517316    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:24.517852 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:24.517445    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:24.517852 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:24.517439    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:24.517852 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:24.517568    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:24.517852 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:24.517647    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:26.517383 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:26.517356    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:26.517383 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:26.517371    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:26.517871 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:26.517459    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txdf4" podUID="074a9efa-d51e-4192-841c-3cd308d89945"
Nov 03 19:26:26.517871 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:26.517538    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfbpg" podUID="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8"
Nov 03 19:26:26.517871 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:26.517549    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:26.517871 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:26.517638    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2l9ws" podUID="f21610fc-0c1e-4ba3-b54e-d3db441d629f"
Nov 03 19:26:27.406028 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.406000    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeReady"
Nov 03 19:26:27.406170 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.406138    2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 03 19:26:27.442013 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.441983    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-ctdfb"]
Nov 03 19:26:27.445254 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.445234    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-ctdfb"
Nov 03 19:26:27.445374 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.445286    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9cxfx"]
Nov 03 19:26:27.448951 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.448932    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pxn4t"]
Nov 03 19:26:27.449082 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.449035    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-9cxfx"
Nov 03 19:26:27.452491 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.452338    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"]
Nov 03 19:26:27.452585 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.452452    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-pxn4t"
Nov 03 19:26:27.455654 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.455636    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"]
Nov 03 19:26:27.455805 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.455788    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:27.459071 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.459029    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"]
Nov 03 19:26:27.459156 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.459110    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:27.462357 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.462341    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-2qjr4"]
Nov 03 19:26:27.462445 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.462437    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:27.466646 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.466590    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-687ff884df-rq9lk"]
Nov 03 19:26:27.466834 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.466654    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.470332 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.470312    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 03 19:26:27.470332 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.470369    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 03 19:26:27.471858 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.471067    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 03 19:26:27.474238 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.474221    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d96b8f96d-xh9sn"]
Nov 03 19:26:27.474378 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.474358    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.475304 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.475286    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 03 19:26:27.475420 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.475342    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-lvj44\""
Nov 03 19:26:27.475606 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.475385    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-dd4dw\""
Nov 03 19:26:27.475606 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.475388    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 03 19:26:27.477690 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.477635    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"]
Nov 03 19:26:27.477981 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.477961    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.480844 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.480822    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"]
Nov 03 19:26:27.481151 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.481136    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:27.485409 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.485386    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"]
Nov 03 19:26:27.486323 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.486278    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:27.490814 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.489349    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9"]
Nov 03 19:26:27.490814 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.489532    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.495999 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.495982    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7"]
Nov 03 19:26:27.496352 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.496333    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9"
Nov 03 19:26:27.500085 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.500066    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"]
Nov 03 19:26:27.500177 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.500136    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7"
Nov 03 19:26:27.502971 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.502954    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-ctdfb"]
Nov 03 19:26:27.503085 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.502987    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9cxfx"]
Nov 03 19:26:27.503085 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.503010    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pxn4t"]
Nov 03 19:26:27.503085 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.503034    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"]
Nov 03 19:26:27.503085 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.503070    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nlt2r"]
Nov 03 19:26:27.503288 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.503126    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.508231 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.508185    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"]
Nov 03 19:26:27.508358 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.508271    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-2qjr4"]
Nov 03 19:26:27.508358 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.508299    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"]
Nov 03 19:26:27.508358 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.508321    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-687ff884df-rq9lk"]
Nov 03 19:26:27.508358 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.508345    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g9g77"]
Nov 03 19:26:27.508530 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.508476    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.511546 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.511526    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7"]
Nov 03 19:26:27.511647 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.511559    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"]
Nov 03 19:26:27.511647 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.511578    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nlt2r"]
Nov 03 19:26:27.511647 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.511600    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"]
Nov 03 19:26:27.511647 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.511615    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"]
Nov 03 19:26:27.511647 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.511631    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d96b8f96d-xh9sn"]
Nov 03 19:26:27.511902 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.511658    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:27.511902 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.511672    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9"]
Nov 03 19:26:27.511902 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.511693    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"]
Nov 03 19:26:27.511902 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.511706    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g9g77"]
Nov 03 19:26:27.567816 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.567798    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 03 19:26:27.568189 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568017    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:27.568347 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568330    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 19:26:27.568412 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568347    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:27.568475 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568415    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 03 19:26:27.568475 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568424    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 03 19:26:27.568475 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568431    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 19:26:27.568475 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568459    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 19:26:27.568475 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568464    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 19:26:27.568475 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568474    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:27.568661 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568494    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 19:26:27.568661 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568498    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 19:26:27.568661 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568506    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 03 19:26:27.568816 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568799    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 19:26:27.568816 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568810    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 19:26:27.568963 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568847    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 03 19:26:27.568963 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568891    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:27.568963 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568910    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 19:26:27.568963 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568932    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 03 19:26:27.568963 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568932    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 03 19:26:27.568963 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568954    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:27.569184 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568991    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:27.569184 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.568991    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:27.570786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.570770    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 03 19:26:27.570786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.570771    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:27.570889 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.570791    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:27.571248 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.571235    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:27.571294 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.571241    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 03 19:26:27.582348 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.582332    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 03 19:26:27.585119 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.584657    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2bmc\" (UniqueName: \"kubernetes.io/projected/68217d54-7abd-4c3f-8aef-3dc2979c1014-kube-api-access-h2bmc\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:27.586138 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586119    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.586216 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586158    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-installation-pull-secrets\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.586216 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586187    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:27.586216 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586212    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vdcm\" (UniqueName: \"kubernetes.io/projected/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-kube-api-access-6vdcm\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:27.586314 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586238    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grqfk\" (UniqueName: \"kubernetes.io/projected/b638dbdf-a1bf-4271-be87-4937d20d4a61-kube-api-access-grqfk\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.586314 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586282    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvxhj\" (UniqueName: \"kubernetes.io/projected/10683ad8-f14a-4034-9768-bb8b168141d1-kube-api-access-jvxhj\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.586371 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586329    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.586371 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586351    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.586435 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586372    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-stats-auth\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.586435 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586393    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-certificates\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.586435 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586412    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-serving-cert\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.586847 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586827    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/13fdaf11-64d1-445d-b055-fbec6b90c1bb-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:27.586921 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586862    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4w5n\" (UniqueName: \"kubernetes.io/projected/8529580b-2186-4766-99d3-b55374b668ef-kube-api-access-j4w5n\") pod \"kas-connection-checker-685895c7c9-9cxfx\" (UID: \"8529580b-2186-4766-99d3-b55374b668ef\") " pod="kube-system/kas-connection-checker-685895c7c9-9cxfx"
Nov 03 19:26:27.586921 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586890    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:27.587012 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586919    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfdk9\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-kube-api-access-rfdk9\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.587012 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586944    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-config\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:27.587012 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.586967    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-trusted-ca\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.587181 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587099    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5tfw\" (UniqueName: \"kubernetes.io/projected/c5f64340-0f40-47b9-949c-6f25f4ef05ab-kube-api-access-l5tfw\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.587181 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587137    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjt5p\" (UniqueName: \"kubernetes.io/projected/ced54a02-fff8-43e5-bebd-80a41ed74f0b-kube-api-access-rjt5p\") pod \"network-check-source-6f78d59f99-sfhm7\" (UID: \"ced54a02-fff8-43e5-bebd-80a41ed74f0b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7"
Nov 03 19:26:27.587181 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587163    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kldqf\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-kube-api-access-kldqf\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.587292 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587188    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-default-certificate\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.587292 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587250    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5f64340-0f40-47b9-949c-6f25f4ef05ab-serving-cert\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.587365 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587293    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-trusted-ca\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.587365 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587330    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59hhh\" (UniqueName: \"kubernetes.io/projected/dd1caca1-f187-4c53-860b-5443c3b95924-kube-api-access-59hhh\") pod \"volume-data-source-validator-58c9f5f46d-4gdd9\" (UID: \"dd1caca1-f187-4c53-860b-5443c3b95924\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9"
Nov 03 19:26:27.587365 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587360    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2mgz\" (UniqueName: \"kubernetes.io/projected/d3f48c1f-1aad-49d7-9063-f6cf60c43907-kube-api-access-v2mgz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:27.587481 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587410    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-image-registry-private-configuration\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.587481 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587445    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.587582 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587514    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:27.587615 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587582    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3f48c1f-1aad-49d7-9063-f6cf60c43907-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:27.587654 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587614    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fjn6\" (UniqueName: \"kubernetes.io/projected/bf43b3af-6048-4d1b-9a8c-96efd387cfc4-kube-api-access-7fjn6\") pod \"kas-connection-checker-685895c7c9-ctdfb\" (UID: \"bf43b3af-6048-4d1b-9a8c-96efd387cfc4\") " pod="kube-system/kas-connection-checker-685895c7c9-ctdfb"
Nov 03 19:26:27.587654 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587641    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d145360-58fb-4376-8126-7933a7abaae2-cert\") pod \"ingress-canary-g9g77\" (UID: \"4d145360-58fb-4376-8126-7933a7abaae2\") " pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:27.587754 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b87d8\" (UniqueName: \"kubernetes.io/projected/13fdaf11-64d1-445d-b055-fbec6b90c1bb-kube-api-access-b87d8\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:27.587754 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587724    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6r2k\" (UniqueName: \"kubernetes.io/projected/4d145360-58fb-4376-8126-7933a7abaae2-kube-api-access-f6r2k\") pod \"ingress-canary-g9g77\" (UID: \"4d145360-58fb-4376-8126-7933a7abaae2\") " pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:27.587843 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587752    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/10683ad8-f14a-4034-9768-bb8b168141d1-config-volume\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.587843 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587775    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10683ad8-f14a-4034-9768-bb8b168141d1-tmp-dir\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.587843 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587829    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-installation-pull-secrets\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.587947 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587846    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ded5b685-660b-4d0f-8100-9c364ab50c7f-ca-trust-extracted\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.587947 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587861    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-certificates\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.587947 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587878    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.587947 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:27.587947 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587928    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51919e3c-99a2-484b-96ac-35948a9eec74-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:27.587947 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587942    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3f48c1f-1aad-49d7-9063-f6cf60c43907-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:27.588142 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.587994    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p7nx\" (UniqueName: \"kubernetes.io/projected/563541bc-4a0f-4e6e-b6f0-ebd9264f537d-kube-api-access-9p7nx\") pod \"kas-connection-checker-685895c7c9-pxn4t\" (UID: \"563541bc-4a0f-4e6e-b6f0-ebd9264f537d\") " pod="kube-system/kas-connection-checker-685895c7c9-pxn4t"
Nov 03 19:26:27.588142 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588019    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-bound-sa-token\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.588142 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588072    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a695e5-8735-4bad-8903-13c26e5bf74a-ca-trust-extracted\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.588238 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588143    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-tmp\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.588238 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588164    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-serving-cert\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:27.588238 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588215    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-config\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.588339 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588245    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbj7f\" (UniqueName: \"kubernetes.io/projected/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-kube-api-access-zbj7f\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.588339 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588264    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.588339 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588309    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-bound-sa-token\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.588339 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588327    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-service-ca-bundle\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.588512 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588370    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-image-registry-private-configuration\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.588512 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588422    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-trusted-ca\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.588512 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588450    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-snapshots\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.588849 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.588823    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 03 19:26:27.590329 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.590310    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 03 19:26:27.590413 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.590318    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 19:26:27.590413 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.590386    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 19:26:27.591320 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.591306    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 03 19:26:27.591409 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.591393    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 03 19:26:27.591850 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.591836    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 03 19:26:27.592786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.592773    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 03 19:26:27.593079 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.593065    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 03 19:26:27.593135 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.593079    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 03 19:26:27.593135 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.593089    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 03 19:26:27.593628 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.593616    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gwvrm\""
Nov 03 19:26:27.593628 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.593624    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7rcqj\""
Nov 03 19:26:27.593687 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.593633    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cj2kt\""
Nov 03 19:26:27.594598 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.594586    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-qhgjf\""
Nov 03 19:26:27.594636 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.594587    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6b2s9\""
Nov 03 19:26:27.594744 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.594733    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-4p8wb\""
Nov 03 19:26:27.594775 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.594734    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ngz6s\""
Nov 03 19:26:27.595411 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.595396    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-td4dq\""
Nov 03 19:26:27.595896 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.595880    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r8qzd\""
Nov 03 19:26:27.596026 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.596012    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wpwwc\""
Nov 03 19:26:27.596083 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.596022    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vjhxp\""
Nov 03 19:26:27.598641 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.598626    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 03 19:26:27.599076 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.599062    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 03 19:26:27.599161 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.599079    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 03 19:26:27.599587 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.599570    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 03 19:26:27.689737 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.689673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p7nx\" (UniqueName: \"kubernetes.io/projected/563541bc-4a0f-4e6e-b6f0-ebd9264f537d-kube-api-access-9p7nx\") pod \"kas-connection-checker-685895c7c9-pxn4t\" (UID: \"563541bc-4a0f-4e6e-b6f0-ebd9264f537d\") " pod="kube-system/kas-connection-checker-685895c7c9-pxn4t"
Nov 03 19:26:27.689737 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.689701    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-bound-sa-token\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.689737 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.689720    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a695e5-8735-4bad-8903-13c26e5bf74a-ca-trust-extracted\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.689962 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.689752    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-tmp\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.689962 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.689952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-serving-cert\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:27.690098 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.689983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-config\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.690098 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690011    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbj7f\" (UniqueName: \"kubernetes.io/projected/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-kube-api-access-zbj7f\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.690098 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690027    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.690254 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690129    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-tmp\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.690254 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.690168    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 19:26:27.690254 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.690229    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls podName:10683ad8-f14a-4034-9768-bb8b168141d1 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:28.190211777 +0000 UTC m=+30.124749562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls") pod "dns-default-nlt2r" (UID: "10683ad8-f14a-4034-9768-bb8b168141d1") : secret "dns-default-metrics-tls" not found
Nov 03 19:26:27.690395 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690271    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a695e5-8735-4bad-8903-13c26e5bf74a-ca-trust-extracted\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.690713 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690691    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-config\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.690797 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690068    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-bound-sa-token\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.690858 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690834    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-service-ca-bundle\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.690912 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690870    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-image-registry-private-configuration\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.690912 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-trusted-ca\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.691008 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690916    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-snapshots\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.691008 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2bmc\" (UniqueName: \"kubernetes.io/projected/68217d54-7abd-4c3f-8aef-3dc2979c1014-kube-api-access-h2bmc\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:27.691008 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.690979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.691172 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691008    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-installation-pull-secrets\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.691172 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691034    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:27.691172 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691078    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vdcm\" (UniqueName: \"kubernetes.io/projected/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-kube-api-access-6vdcm\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:27.691172 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691107    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grqfk\" (UniqueName: \"kubernetes.io/projected/b638dbdf-a1bf-4271-be87-4937d20d4a61-kube-api-access-grqfk\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.691172 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691131    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvxhj\" (UniqueName: \"kubernetes.io/projected/10683ad8-f14a-4034-9768-bb8b168141d1-kube-api-access-jvxhj\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.691172 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691181    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691203    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-stats-auth\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691240    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-certificates\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691264    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-serving-cert\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691289    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/13fdaf11-64d1-445d-b055-fbec6b90c1bb-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691316    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4w5n\" (UniqueName: \"kubernetes.io/projected/8529580b-2186-4766-99d3-b55374b668ef-kube-api-access-j4w5n\") pod \"kas-connection-checker-685895c7c9-9cxfx\" (UID: \"8529580b-2186-4766-99d3-b55374b668ef\") " pod="kube-system/kas-connection-checker-685895c7c9-9cxfx"
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691343    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691356    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-service-ca-bundle\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691369    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfdk9\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-kube-api-access-rfdk9\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691397    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-config\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.691321    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.691371    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.691430    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r: secret "image-registry-tls" not found
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.691478    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.691505    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls podName:ded5b685-660b-4d0f-8100-9c364ab50c7f nodeName:}" failed. No retries permitted until 2026-11-03 19:26:28.19148743 +0000 UTC m=+30.126025200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls") pod "image-registry-7cc7bcdcc7-r2w7r" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f") : secret "image-registry-tls" not found
Nov 03 19:26:27.691623 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691524    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-trusted-ca\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.691537    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert podName:51919e3c-99a2-484b-96ac-35948a9eec74 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:28.191521881 +0000 UTC m=+30.126059673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pv5k4" (UID: "51919e3c-99a2-484b-96ac-35948a9eec74") : secret "networking-console-plugin-cert" not found
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5tfw\" (UniqueName: \"kubernetes.io/projected/c5f64340-0f40-47b9-949c-6f25f4ef05ab-kube-api-access-l5tfw\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjt5p\" (UniqueName: \"kubernetes.io/projected/ced54a02-fff8-43e5-bebd-80a41ed74f0b-kube-api-access-rjt5p\") pod \"network-check-source-6f78d59f99-sfhm7\" (UID: \"ced54a02-fff8-43e5-bebd-80a41ed74f0b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691617    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kldqf\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-kube-api-access-kldqf\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691638    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-default-certificate\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691667    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5f64340-0f40-47b9-949c-6f25f4ef05ab-serving-cert\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691694    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-trusted-ca\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691719    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59hhh\" (UniqueName: \"kubernetes.io/projected/dd1caca1-f187-4c53-860b-5443c3b95924-kube-api-access-59hhh\") pod \"volume-data-source-validator-58c9f5f46d-4gdd9\" (UID: \"dd1caca1-f187-4c53-860b-5443c3b95924\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691745    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2mgz\" (UniqueName: \"kubernetes.io/projected/d3f48c1f-1aad-49d7-9063-f6cf60c43907-kube-api-access-v2mgz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691773    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-image-registry-private-configuration\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691801    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691827    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691850    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3f48c1f-1aad-49d7-9063-f6cf60c43907-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:27.692402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691873    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fjn6\" (UniqueName: \"kubernetes.io/projected/bf43b3af-6048-4d1b-9a8c-96efd387cfc4-kube-api-access-7fjn6\") pod \"kas-connection-checker-685895c7c9-ctdfb\" (UID: \"bf43b3af-6048-4d1b-9a8c-96efd387cfc4\") " pod="kube-system/kas-connection-checker-685895c7c9-ctdfb"
Nov 03 19:26:27.693113 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691901    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d145360-58fb-4376-8126-7933a7abaae2-cert\") pod \"ingress-canary-g9g77\" (UID: \"4d145360-58fb-4376-8126-7933a7abaae2\") " pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:27.693113 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691926    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b87d8\" (UniqueName: \"kubernetes.io/projected/13fdaf11-64d1-445d-b055-fbec6b90c1bb-kube-api-access-b87d8\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:27.693113 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691956    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6r2k\" (UniqueName: \"kubernetes.io/projected/4d145360-58fb-4376-8126-7933a7abaae2-kube-api-access-f6r2k\") pod \"ingress-canary-g9g77\" (UID: \"4d145360-58fb-4376-8126-7933a7abaae2\") " pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:27.693113 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/10683ad8-f14a-4034-9768-bb8b168141d1-config-volume\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.693113 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.691999    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10683ad8-f14a-4034-9768-bb8b168141d1-tmp-dir\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.693113 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.692060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-installation-pull-secrets\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.693113 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.692072    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-config\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.692098    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ded5b685-660b-4d0f-8100-9c364ab50c7f-ca-trust-extracted\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.692383    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-trusted-ca\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.692440    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ded5b685-660b-4d0f-8100-9c364ab50c7f-ca-trust-extracted\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.692515    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.694794    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d96b8f96d-xh9sn: secret "image-registry-tls" not found
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.692793    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/13fdaf11-64d1-445d-b055-fbec6b90c1bb-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.694140    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.692579    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.693113    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3f48c1f-1aad-49d7-9063-f6cf60c43907-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.694733    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle podName:b638dbdf-a1bf-4271-be87-4937d20d4a61 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:28.194717035 +0000 UTC m=+30.129254817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle") pod "router-default-687ff884df-rq9lk" (UID: "b638dbdf-a1bf-4271-be87-4937d20d4a61") : configmap references non-existent config key: service-ca.crt
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.694910    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls podName:d4a695e5-8735-4bad-8903-13c26e5bf74a nodeName:}" failed. No retries permitted until 2026-11-03 19:26:28.194897749 +0000 UTC m=+30.129435520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls") pod "image-registry-7d96b8f96d-xh9sn" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a") : secret "image-registry-tls" not found
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.694925    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls podName:68217d54-7abd-4c3f-8aef-3dc2979c1014 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:28.194916289 +0000 UTC m=+30.129454055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6gdg5" (UID: "68217d54-7abd-4c3f-8aef-3dc2979c1014") : secret "samples-operator-tls" not found
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.694945    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-certificates\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.694964    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10683ad8-f14a-4034-9768-bb8b168141d1-tmp-dir\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.694970    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-trusted-ca\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.697195 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.695284    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-snapshots\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.695475    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-certificates\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.695551    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.695580    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.695622    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51919e3c-99a2-484b-96ac-35948a9eec74-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.695647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3f48c1f-1aad-49d7-9063-f6cf60c43907-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.695668    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/10683ad8-f14a-4034-9768-bb8b168141d1-config-volume\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.696258    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-installation-pull-secrets\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.696269    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d145360-58fb-4376-8126-7933a7abaae2-cert\") pod \"ingress-canary-g9g77\" (UID: \"4d145360-58fb-4376-8126-7933a7abaae2\") " pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.696463    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-serving-cert\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.696719    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls podName:13fdaf11-64d1-445d-b055-fbec6b90c1bb nodeName:}" failed. No retries permitted until 2026-11-03 19:26:28.196700655 +0000 UTC m=+30.131238441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-x8xpg" (UID: "13fdaf11-64d1-445d-b055-fbec6b90c1bb") : secret "cluster-monitoring-operator-tls" not found
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.696960    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-stats-auth\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.697184    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-serving-cert\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.698033 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.697215    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-certificates\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.698694 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.698559    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-image-registry-private-configuration\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.699627 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.699022    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:27.699627 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.699176    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 03 19:26:27.699627 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:27.699246    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs podName:b638dbdf-a1bf-4271-be87-4937d20d4a61 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:28.199232795 +0000 UTC m=+30.133770565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs") pod "router-default-687ff884df-rq9lk" (UID: "b638dbdf-a1bf-4271-be87-4937d20d4a61") : secret "router-metrics-certs-default" not found
Nov 03 19:26:27.699627 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.699245    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-installation-pull-secrets\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.699627 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.699357    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51919e3c-99a2-484b-96ac-35948a9eec74-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:27.699627 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.699553    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-default-certificate\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.700009 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.699671    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3f48c1f-1aad-49d7-9063-f6cf60c43907-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:27.700009 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.699943    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-image-registry-private-configuration\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.700183 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.700161    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-trusted-ca\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.701480 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.701456    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5f64340-0f40-47b9-949c-6f25f4ef05ab-serving-cert\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.706928 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.706906    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-bound-sa-token\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.710518 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.709847    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vdcm\" (UniqueName: \"kubernetes.io/projected/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-kube-api-access-6vdcm\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:27.710518 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.709872    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvxhj\" (UniqueName: \"kubernetes.io/projected/10683ad8-f14a-4034-9768-bb8b168141d1-kube-api-access-jvxhj\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:27.710518 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.710401    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2bmc\" (UniqueName: \"kubernetes.io/projected/68217d54-7abd-4c3f-8aef-3dc2979c1014-kube-api-access-h2bmc\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:27.710518 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.710440    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6r2k\" (UniqueName: \"kubernetes.io/projected/4d145360-58fb-4376-8126-7933a7abaae2-kube-api-access-f6r2k\") pod \"ingress-canary-g9g77\" (UID: \"4d145360-58fb-4376-8126-7933a7abaae2\") " pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:27.710835 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.710813    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fjn6\" (UniqueName: \"kubernetes.io/projected/bf43b3af-6048-4d1b-9a8c-96efd387cfc4-kube-api-access-7fjn6\") pod \"kas-connection-checker-685895c7c9-ctdfb\" (UID: \"bf43b3af-6048-4d1b-9a8c-96efd387cfc4\") " pod="kube-system/kas-connection-checker-685895c7c9-ctdfb"
Nov 03 19:26:27.711060 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.711023    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-bound-sa-token\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.711641 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.711541    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4w5n\" (UniqueName: \"kubernetes.io/projected/8529580b-2186-4766-99d3-b55374b668ef-kube-api-access-j4w5n\") pod \"kas-connection-checker-685895c7c9-9cxfx\" (UID: \"8529580b-2186-4766-99d3-b55374b668ef\") " pod="kube-system/kas-connection-checker-685895c7c9-9cxfx"
Nov 03 19:26:27.711641 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.711579    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b87d8\" (UniqueName: \"kubernetes.io/projected/13fdaf11-64d1-445d-b055-fbec6b90c1bb-kube-api-access-b87d8\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:27.711760 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.711708    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfdk9\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-kube-api-access-rfdk9\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:27.712613 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.712588    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbj7f\" (UniqueName: \"kubernetes.io/projected/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-kube-api-access-zbj7f\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.713150 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.713127    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p7nx\" (UniqueName: \"kubernetes.io/projected/563541bc-4a0f-4e6e-b6f0-ebd9264f537d-kube-api-access-9p7nx\") pod \"kas-connection-checker-685895c7c9-pxn4t\" (UID: \"563541bc-4a0f-4e6e-b6f0-ebd9264f537d\") " pod="kube-system/kas-connection-checker-685895c7c9-pxn4t"
Nov 03 19:26:27.714458 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.714433    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjt5p\" (UniqueName: \"kubernetes.io/projected/ced54a02-fff8-43e5-bebd-80a41ed74f0b-kube-api-access-rjt5p\") pod \"network-check-source-6f78d59f99-sfhm7\" (UID: \"ced54a02-fff8-43e5-bebd-80a41ed74f0b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7"
Nov 03 19:26:27.714616 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.714593    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kldqf\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-kube-api-access-kldqf\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:27.716189 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.716144    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5tfw\" (UniqueName: \"kubernetes.io/projected/c5f64340-0f40-47b9-949c-6f25f4ef05ab-kube-api-access-l5tfw\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.717092 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.716593    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59hhh\" (UniqueName: \"kubernetes.io/projected/dd1caca1-f187-4c53-860b-5443c3b95924-kube-api-access-59hhh\") pod \"volume-data-source-validator-58c9f5f46d-4gdd9\" (UID: \"dd1caca1-f187-4c53-860b-5443c3b95924\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9"
Nov 03 19:26:27.717092 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.717021    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grqfk\" (UniqueName: \"kubernetes.io/projected/b638dbdf-a1bf-4271-be87-4937d20d4a61-kube-api-access-grqfk\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:27.717774 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.717752    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2mgz\" (UniqueName: \"kubernetes.io/projected/d3f48c1f-1aad-49d7-9063-f6cf60c43907-kube-api-access-v2mgz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:27.760724 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.760691    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-ctdfb"
Nov 03 19:26:27.765916 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.765895    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-9cxfx"
Nov 03 19:26:27.774939 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.774913    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-pxn4t"
Nov 03 19:26:27.787781 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.787758    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:27.807402 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.807161    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:27.813733 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.813710    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:27.843792 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.843709    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:27.852010 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.851601    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9"
Nov 03 19:26:27.857967 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.857669    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7"
Nov 03 19:26:27.879010 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:27.878603    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:28.067991 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.067970    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9cxfx"]
Nov 03 19:26:28.070547 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:28.070508    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8529580b_2186_4766_99d3_b55374b668ef.slice/crio-f56aa45e661ed322b53efdbdba9ccccc5bebd1b1a87ec583b79c6b6f0b911383 WatchSource:0}: Error finding container f56aa45e661ed322b53efdbdba9ccccc5bebd1b1a87ec583b79c6b6f0b911383: Status 404 returned error can't find the container with id f56aa45e661ed322b53efdbdba9ccccc5bebd1b1a87ec583b79c6b6f0b911383
Nov 03 19:26:28.070981 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.070959    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-ctdfb"]
Nov 03 19:26:28.073793 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:28.073395    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf43b3af_6048_4d1b_9a8c_96efd387cfc4.slice/crio-6f37acc05f224bec8051f437f9347b9f93f9ed1b29430c7f570688d6202e7e2c WatchSource:0}: Error finding container 6f37acc05f224bec8051f437f9347b9f93f9ed1b29430c7f570688d6202e7e2c: Status 404 returned error can't find the container with id 6f37acc05f224bec8051f437f9347b9f93f9ed1b29430c7f570688d6202e7e2c
Nov 03 19:26:28.074385 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.074362    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pxn4t"]
Nov 03 19:26:28.080640 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:28.080615    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563541bc_4a0f_4e6e_b6f0_ebd9264f537d.slice/crio-9da85617cf620e26573ebd5f669f772f0a1ce4adc25ddd9c0c88bed500e72fed WatchSource:0}: Error finding container 9da85617cf620e26573ebd5f669f772f0a1ce4adc25ddd9c0c88bed500e72fed: Status 404 returned error can't find the container with id 9da85617cf620e26573ebd5f669f772f0a1ce4adc25ddd9c0c88bed500e72fed
Nov 03 19:26:28.100890 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.100809    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"]
Nov 03 19:26:28.143098 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.143075    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-2qjr4"]
Nov 03 19:26:28.146665 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.146635    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"]
Nov 03 19:26:28.178838 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.176789    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"]
Nov 03 19:26:28.187899 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:28.187876    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5f64340_0f40_47b9_949c_6f25f4ef05ab.slice/crio-e3c3c69d9cc25f34a9cbb0dd34b77aafb3a2a1ffd01c5569992068d6c8cf96f2 WatchSource:0}: Error finding container e3c3c69d9cc25f34a9cbb0dd34b77aafb3a2a1ffd01c5569992068d6c8cf96f2: Status 404 returned error can't find the container with id e3c3c69d9cc25f34a9cbb0dd34b77aafb3a2a1ffd01c5569992068d6c8cf96f2
Nov 03 19:26:28.196477 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.196455    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7"]
Nov 03 19:26:28.201602 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:28.201579    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced54a02_fff8_43e5_bebd_80a41ed74f0b.slice/crio-b220883be33b4038b1c0da7eac6ff4d8c7afee341d75efb68fc7e698bc2bd4ac WatchSource:0}: Error finding container b220883be33b4038b1c0da7eac6ff4d8c7afee341d75efb68fc7e698bc2bd4ac: Status 404 returned error can't find the container with id b220883be33b4038b1c0da7eac6ff4d8c7afee341d75efb68fc7e698bc2bd4ac
Nov 03 19:26:28.207968 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.207943    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:28.208108 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.207993    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:28.208108 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.208023    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:28.208108 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.208068    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:28.208108 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.208102    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:28.208292 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.208154    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:28.208292 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.208186    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:28.208292 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.208203    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:28.208405 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208324    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 03 19:26:28.208405 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208380    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls podName:13fdaf11-64d1-445d-b055-fbec6b90c1bb nodeName:}" failed. No retries permitted until 2026-11-03 19:26:29.2083605 +0000 UTC m=+31.142898281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-x8xpg" (UID: "13fdaf11-64d1-445d-b055-fbec6b90c1bb") : secret "cluster-monitoring-operator-tls" not found
Nov 03 19:26:28.208481 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208441    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle podName:b638dbdf-a1bf-4271-be87-4937d20d4a61 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:29.208434492 +0000 UTC m=+31.142972257 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle") pod "router-default-687ff884df-rq9lk" (UID: "b638dbdf-a1bf-4271-be87-4937d20d4a61") : configmap references non-existent config key: service-ca.crt
Nov 03 19:26:28.208543 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208501    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 03 19:26:28.208543 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208522    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert podName:51919e3c-99a2-484b-96ac-35948a9eec74 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:29.2085158 +0000 UTC m=+31.143053565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pv5k4" (UID: "51919e3c-99a2-484b-96ac-35948a9eec74") : secret "networking-console-plugin-cert" not found
Nov 03 19:26:28.208648 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208564    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 19:26:28.208648 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208571    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d96b8f96d-xh9sn: secret "image-registry-tls" not found
Nov 03 19:26:28.208648 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208590    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls podName:d4a695e5-8735-4bad-8903-13c26e5bf74a nodeName:}" failed. No retries permitted until 2026-11-03 19:26:29.208584884 +0000 UTC m=+31.143122649 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls") pod "image-registry-7d96b8f96d-xh9sn" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a") : secret "image-registry-tls" not found
Nov 03 19:26:28.208648 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208625    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 03 19:26:28.208648 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208644    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls podName:68217d54-7abd-4c3f-8aef-3dc2979c1014 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:29.208638341 +0000 UTC m=+31.143176106 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6gdg5" (UID: "68217d54-7abd-4c3f-8aef-3dc2979c1014") : secret "samples-operator-tls" not found
Nov 03 19:26:28.208867 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208676    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 03 19:26:28.208867 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208700    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs podName:b638dbdf-a1bf-4271-be87-4937d20d4a61 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:29.208692684 +0000 UTC m=+31.143230455 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs") pod "router-default-687ff884df-rq9lk" (UID: "b638dbdf-a1bf-4271-be87-4937d20d4a61") : secret "router-metrics-certs-default" not found
Nov 03 19:26:28.208867 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208747    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 19:26:28.208867 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208776    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls podName:10683ad8-f14a-4034-9768-bb8b168141d1 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:29.208766921 +0000 UTC m=+31.143304686 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls") pod "dns-default-nlt2r" (UID: "10683ad8-f14a-4034-9768-bb8b168141d1") : secret "dns-default-metrics-tls" not found
Nov 03 19:26:28.208867 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208839    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 19:26:28.208867 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208850    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r: secret "image-registry-tls" not found
Nov 03 19:26:28.209054 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:28.208877    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls podName:ded5b685-660b-4d0f-8100-9c364ab50c7f nodeName:}" failed. No retries permitted until 2026-11-03 19:26:29.208868936 +0000 UTC m=+31.143406715 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls") pod "image-registry-7cc7bcdcc7-r2w7r" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f") : secret "image-registry-tls" not found
Nov 03 19:26:28.227371 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.227342    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9"]
Nov 03 19:26:28.228730 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:28.228709    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd1caca1_f187_4c53_860b_5443c3b95924.slice/crio-8b2d52d95416e8b2657ace8a275373d23433fcc6b43c9ebe0d63d86c99966199 WatchSource:0}: Error finding container 8b2d52d95416e8b2657ace8a275373d23433fcc6b43c9ebe0d63d86c99966199: Status 404 returned error can't find the container with id 8b2d52d95416e8b2657ace8a275373d23433fcc6b43c9ebe0d63d86c99966199
Nov 03 19:26:28.238185 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.238157    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g9g77"]
Nov 03 19:26:28.240207 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:28.240185    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d145360_58fb_4376_8126_7933a7abaae2.slice/crio-810344098a090bd63f39706afcf9d9585eb33a03429273a625d68f898c17190c WatchSource:0}: Error finding container 810344098a090bd63f39706afcf9d9585eb33a03429273a625d68f898c17190c: Status 404 returned error can't find the container with id 810344098a090bd63f39706afcf9d9585eb33a03429273a625d68f898c17190c
Nov 03 19:26:28.518158 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.518132    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:28.518301 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.518217    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:28.518301 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.518261    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:28.523690 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.523429    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 19:26:28.523690 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.523552    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-xlp4b\""
Nov 03 19:26:28.523690 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.523554    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xv5d7\""
Nov 03 19:26:28.523690 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.523640    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 19:26:28.523947 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.523692    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-5vlcb\""
Nov 03 19:26:28.778263 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.778185    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7" event={"ID":"ced54a02-fff8-43e5-bebd-80a41ed74f0b","Type":"ContainerStarted","Data":"b220883be33b4038b1c0da7eac6ff4d8c7afee341d75efb68fc7e698bc2bd4ac"}
Nov 03 19:26:28.779771 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.779747    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9" event={"ID":"dd1caca1-f187-4c53-860b-5443c3b95924","Type":"ContainerStarted","Data":"8b2d52d95416e8b2657ace8a275373d23433fcc6b43c9ebe0d63d86c99966199"}
Nov 03 19:26:28.784112 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.784034    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-pxn4t" event={"ID":"563541bc-4a0f-4e6e-b6f0-ebd9264f537d","Type":"ContainerStarted","Data":"e0e1ed1c9f43743b15fda9718b609f782f3cfb8cc083b3c38970242d21d17bef"}
Nov 03 19:26:28.784215 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.784134    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-pxn4t" event={"ID":"563541bc-4a0f-4e6e-b6f0-ebd9264f537d","Type":"ContainerStarted","Data":"9da85617cf620e26573ebd5f669f772f0a1ce4adc25ddd9c0c88bed500e72fed"}
Nov 03 19:26:28.785530 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.785505    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj" event={"ID":"d3f48c1f-1aad-49d7-9063-f6cf60c43907","Type":"ContainerStarted","Data":"5814ab61a89b534521e6eb380fe73e30473b71a8bd33e618c7c50d4970a0e85c"}
Nov 03 19:26:28.787970 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.787951    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-9cxfx" event={"ID":"8529580b-2186-4766-99d3-b55374b668ef","Type":"ContainerStarted","Data":"e74f5c72a487e77a180e1f953e9ead6e17ac93e3479e05ee266996e1d14b75ec"}
Nov 03 19:26:28.788068 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.787984    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-9cxfx" event={"ID":"8529580b-2186-4766-99d3-b55374b668ef","Type":"ContainerStarted","Data":"f56aa45e661ed322b53efdbdba9ccccc5bebd1b1a87ec583b79c6b6f0b911383"}
Nov 03 19:26:28.790629 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.790189    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-ctdfb" event={"ID":"bf43b3af-6048-4d1b-9a8c-96efd387cfc4","Type":"ContainerStarted","Data":"f5abbca434f4906be0932a4f1e78ac6f8b2f9b12f84b8d30605f64f9d209a0f9"}
Nov 03 19:26:28.790629 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.790228    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-ctdfb" event={"ID":"bf43b3af-6048-4d1b-9a8c-96efd387cfc4","Type":"ContainerStarted","Data":"6f37acc05f224bec8051f437f9347b9f93f9ed1b29430c7f570688d6202e7e2c"}
Nov 03 19:26:28.793877 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.793848    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6" event={"ID":"c5f64340-0f40-47b9-949c-6f25f4ef05ab","Type":"ContainerStarted","Data":"e3c3c69d9cc25f34a9cbb0dd34b77aafb3a2a1ffd01c5569992068d6c8cf96f2"}
Nov 03 19:26:28.797114 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.797087    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g9g77" event={"ID":"4d145360-58fb-4376-8126-7933a7abaae2","Type":"ContainerStarted","Data":"810344098a090bd63f39706afcf9d9585eb33a03429273a625d68f898c17190c"}
Nov 03 19:26:28.800925 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.800902    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-2qjr4" event={"ID":"c4f3fa56-4b85-46bd-8c97-94da036ce8bb","Type":"ContainerStarted","Data":"dc83d44b0f2991e38aa23efc9ff63fd33e647a7ac0d7090a13312cec686f34f9"}
Nov 03 19:26:28.805333 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.803947    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-pxn4t" podStartSLOduration=371.803931456 podStartE2EDuration="6m11.803931456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:20:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:28.80320186 +0000 UTC m=+30.737739647" watchObservedRunningTime="2026-11-03 19:26:28.803931456 +0000 UTC m=+30.738469246"
Nov 03 19:26:28.805333 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.805318    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25" event={"ID":"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd","Type":"ContainerStarted","Data":"b8d25f522b81416b4b2b5eca72086afb0a1674793765958b5caf41553f663e65"}
Nov 03 19:26:28.843091 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.843024    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-ctdfb" podStartSLOduration=371.843008202 podStartE2EDuration="6m11.843008202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:20:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:28.842887567 +0000 UTC m=+30.777425353" watchObservedRunningTime="2026-11-03 19:26:28.843008202 +0000 UTC m=+30.777545980"
Nov 03 19:26:28.844286 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:28.844224    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-9cxfx" podStartSLOduration=371.844208745 podStartE2EDuration="6m11.844208745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:20:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:28.823128682 +0000 UTC m=+30.757666482" watchObservedRunningTime="2026-11-03 19:26:28.844208745 +0000 UTC m=+30.778746533"
Nov 03 19:26:29.219987 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:29.219955    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:29.220174 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:29.220027    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:29.220174 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:29.220068    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:29.220174 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:29.220089    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:29.220174 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:29.220111    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:29.220174 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:29.220135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:29.220174 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:29.220159    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:29.220174 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:29.220175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:29.220509 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220275    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 03 19:26:29.220509 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220322    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls podName:68217d54-7abd-4c3f-8aef-3dc2979c1014 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:31.220310436 +0000 UTC m=+33.154848201 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6gdg5" (UID: "68217d54-7abd-4c3f-8aef-3dc2979c1014") : secret "samples-operator-tls" not found
Nov 03 19:26:29.220651 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220636    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 03 19:26:29.220704 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220675    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs podName:b638dbdf-a1bf-4271-be87-4937d20d4a61 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:31.220665832 +0000 UTC m=+33.155203596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs") pod "router-default-687ff884df-rq9lk" (UID: "b638dbdf-a1bf-4271-be87-4937d20d4a61") : secret "router-metrics-certs-default" not found
Nov 03 19:26:29.220754 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220713    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 19:26:29.220754 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220731    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls podName:10683ad8-f14a-4034-9768-bb8b168141d1 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:31.220725268 +0000 UTC m=+33.155263032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls") pod "dns-default-nlt2r" (UID: "10683ad8-f14a-4034-9768-bb8b168141d1") : secret "dns-default-metrics-tls" not found
Nov 03 19:26:29.220863 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220777    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 19:26:29.220863 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220789    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r: secret "image-registry-tls" not found
Nov 03 19:26:29.220863 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220817    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls podName:ded5b685-660b-4d0f-8100-9c364ab50c7f nodeName:}" failed. No retries permitted until 2026-11-03 19:26:31.220808008 +0000 UTC m=+33.155345778 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls") pod "image-registry-7cc7bcdcc7-r2w7r" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f") : secret "image-registry-tls" not found
Nov 03 19:26:29.221008 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220870    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 03 19:26:29.221008 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220900    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls podName:13fdaf11-64d1-445d-b055-fbec6b90c1bb nodeName:}" failed. No retries permitted until 2026-11-03 19:26:31.22089063 +0000 UTC m=+33.155428402 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-x8xpg" (UID: "13fdaf11-64d1-445d-b055-fbec6b90c1bb") : secret "cluster-monitoring-operator-tls" not found
Nov 03 19:26:29.221008 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.220960    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle podName:b638dbdf-a1bf-4271-be87-4937d20d4a61 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:31.220950877 +0000 UTC m=+33.155488641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle") pod "router-default-687ff884df-rq9lk" (UID: "b638dbdf-a1bf-4271-be87-4937d20d4a61") : configmap references non-existent config key: service-ca.crt
Nov 03 19:26:29.221180 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.221010    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 03 19:26:29.221180 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.221053    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert podName:51919e3c-99a2-484b-96ac-35948a9eec74 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:31.221030218 +0000 UTC m=+33.155567996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pv5k4" (UID: "51919e3c-99a2-484b-96ac-35948a9eec74") : secret "networking-console-plugin-cert" not found
Nov 03 19:26:29.221180 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.221094    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 19:26:29.221180 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.221102    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d96b8f96d-xh9sn: secret "image-registry-tls" not found
Nov 03 19:26:29.221180 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:29.221119    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls podName:d4a695e5-8735-4bad-8903-13c26e5bf74a nodeName:}" failed. No retries permitted until 2026-11-03 19:26:31.221113834 +0000 UTC m=+33.155651598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls") pod "image-registry-7d96b8f96d-xh9sn" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a") : secret "image-registry-tls" not found
Nov 03 19:26:30.821843 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:30.821001    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9" event={"ID":"dd1caca1-f187-4c53-860b-5443c3b95924","Type":"ContainerStarted","Data":"8247ab4def20cf0984e8dc684e235116ce970a2ac5232fbac47953262b6b8bbe"}
Nov 03 19:26:30.841651 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:30.841597    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9" podStartSLOduration=314.095002746 podStartE2EDuration="5m15.84157995s" totalImagesPullingTime="1.746577204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:21:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:30.8411526 +0000 UTC m=+32.775690390" watchObservedRunningTime="2026-11-03 19:26:30.84157995 +0000 UTC m=+32.776117738"
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:31.246786    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:31.246839    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:31.246884    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:31.246925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:31.246965    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:31.246994    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:31.247061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:31.247134    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.247280    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.247356    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls podName:10683ad8-f14a-4034-9768-bb8b168141d1 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:35.247338845 +0000 UTC m=+37.181876614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls") pod "dns-default-nlt2r" (UID: "10683ad8-f14a-4034-9768-bb8b168141d1") : secret "dns-default-metrics-tls" not found
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.247777    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.247793    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r: secret "image-registry-tls" not found
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.247836    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls podName:ded5b685-660b-4d0f-8100-9c364ab50c7f nodeName:}" failed. No retries permitted until 2026-11-03 19:26:35.247822201 +0000 UTC m=+37.182359971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls") pod "image-registry-7cc7bcdcc7-r2w7r" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f") : secret "image-registry-tls" not found
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.247897    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 03 19:26:31.248441 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.247931    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls podName:13fdaf11-64d1-445d-b055-fbec6b90c1bb nodeName:}" failed. No retries permitted until 2026-11-03 19:26:35.247920258 +0000 UTC m=+37.182458028 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-x8xpg" (UID: "13fdaf11-64d1-445d-b055-fbec6b90c1bb") : secret "cluster-monitoring-operator-tls" not found
Nov 03 19:26:31.249387 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.247994    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle podName:b638dbdf-a1bf-4271-be87-4937d20d4a61 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:35.247983181 +0000 UTC m=+37.182520960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle") pod "router-default-687ff884df-rq9lk" (UID: "b638dbdf-a1bf-4271-be87-4937d20d4a61") : configmap references non-existent config key: service-ca.crt
Nov 03 19:26:31.249387 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.248061    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 03 19:26:31.249387 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.248102    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert podName:51919e3c-99a2-484b-96ac-35948a9eec74 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:35.248082922 +0000 UTC m=+37.182620691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pv5k4" (UID: "51919e3c-99a2-484b-96ac-35948a9eec74") : secret "networking-console-plugin-cert" not found
Nov 03 19:26:31.249387 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.248160    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 19:26:31.249387 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.248169    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d96b8f96d-xh9sn: secret "image-registry-tls" not found
Nov 03 19:26:31.249387 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.248199    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls podName:d4a695e5-8735-4bad-8903-13c26e5bf74a nodeName:}" failed. No retries permitted until 2026-11-03 19:26:35.248189157 +0000 UTC m=+37.182726929 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls") pod "image-registry-7d96b8f96d-xh9sn" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a") : secret "image-registry-tls" not found
Nov 03 19:26:31.249387 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.248248    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 03 19:26:31.249387 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.248277    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls podName:68217d54-7abd-4c3f-8aef-3dc2979c1014 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:35.248267707 +0000 UTC m=+37.182805476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6gdg5" (UID: "68217d54-7abd-4c3f-8aef-3dc2979c1014") : secret "samples-operator-tls" not found
Nov 03 19:26:31.249387 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.248332    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 03 19:26:31.249387 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:31.248362    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs podName:b638dbdf-a1bf-4271-be87-4937d20d4a61 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:35.248352376 +0000 UTC m=+37.182890145 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs") pod "router-default-687ff884df-rq9lk" (UID: "b638dbdf-a1bf-4271-be87-4937d20d4a61") : secret "router-metrics-certs-default" not found
Nov 03 19:26:32.359589 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:32.359550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:32.360136 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:32.359610    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:32.360136 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:32.359743    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 03 19:26:32.360136 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:32.359831    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs podName:ac2071b7-e0b9-4761-ae26-bc5c0ee993d8 nodeName:}" failed. No retries permitted until 2026-11-03 19:27:04.359808059 +0000 UTC m=+66.294345852 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs") pod "network-metrics-daemon-jfbpg" (UID: "ac2071b7-e0b9-4761-ae26-bc5c0ee993d8") : secret "metrics-daemon-secret" not found
Nov 03 19:26:32.363809 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:32.363782    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:32.439542 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:32.439509    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:32.460966 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:32.460940    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfg6f\" (UniqueName: \"kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f\") pod \"network-check-target-2l9ws\" (UID: \"f21610fc-0c1e-4ba3-b54e-d3db441d629f\") " pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:32.471424 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:32.471396    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfg6f\" (UniqueName: \"kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f\") pod \"network-check-target-2l9ws\" (UID: \"f21610fc-0c1e-4ba3-b54e-d3db441d629f\") " pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:32.732494 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:32.732314    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:32.828459 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:32.828436    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/0.log"
Nov 03 19:26:32.828546 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:32.828472    2574 generic.go:356] "Generic (PLEG): container finished" podID="c5f64340-0f40-47b9-949c-6f25f4ef05ab" containerID="282d3f9ada3abd8f83b497bfa6fb2498e98ebb9772821316ffca08c796b41261" exitCode=255
Nov 03 19:26:32.828546 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:32.828513    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6" event={"ID":"c5f64340-0f40-47b9-949c-6f25f4ef05ab","Type":"ContainerDied","Data":"282d3f9ada3abd8f83b497bfa6fb2498e98ebb9772821316ffca08c796b41261"}
Nov 03 19:26:32.828866 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:32.828846    2574 scope.go:122] "RemoveContainer" containerID="282d3f9ada3abd8f83b497bfa6fb2498e98ebb9772821316ffca08c796b41261"
Nov 03 19:26:33.076659 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.076626    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2l9ws"]
Nov 03 19:26:33.080753 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.080473    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txdf4"]
Nov 03 19:26:33.166291 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:33.166251    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf21610fc_0c1e_4ba3_b54e_d3db441d629f.slice/crio-1fb59f3014e2bb210aef2057069c03ed78c84a6822166269be221c2393a8b8d3 WatchSource:0}: Error finding container 1fb59f3014e2bb210aef2057069c03ed78c84a6822166269be221c2393a8b8d3: Status 404 returned error can't find the container with id 1fb59f3014e2bb210aef2057069c03ed78c84a6822166269be221c2393a8b8d3
Nov 03 19:26:33.519720 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.519648    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lb4p7"]
Nov 03 19:26:33.539621 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.539589    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lb4p7"]
Nov 03 19:26:33.539786 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.539699    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.541267 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.541246    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-h9gh8-2nbhw"]
Nov 03 19:26:33.560705 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.560684    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-h9gh8-2nbhw"]
Nov 03 19:26:33.560807 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.560770    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.574321 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.574296    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 19:26:33.574438 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.574427    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gtgvx\""
Nov 03 19:26:33.587285 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.587263    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 19:26:33.671787 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.671766    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ae31de6-e744-4e9a-a9dd-7c5203acad17-data-volume\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.671883 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.671840    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.671930 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.671891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.672003 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.671977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8px2r\" (UniqueName: \"kubernetes.io/projected/5f14faa0-564b-4e55-a3bc-984c152c5ee2-kube-api-access-8px2r\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.672079 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.672053    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ae31de6-e744-4e9a-a9dd-7c5203acad17-crio-socket\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.672113 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.672101    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-service-ca-bundle\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.672215 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.672198    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5f14faa0-564b-4e55-a3bc-984c152c5ee2-archives-path\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.672264 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.672254    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-trusted-ca-bundle\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.672329 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.672312    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwfkd\" (UniqueName: \"kubernetes.io/projected/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-api-access-vwfkd\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.753369 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.753341    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr"]
Nov 03 19:26:33.772067 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.771983    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct"]
Nov 03 19:26:33.772183 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.772128    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr"
Nov 03 19:26:33.772897 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.772873    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8px2r\" (UniqueName: \"kubernetes.io/projected/5f14faa0-564b-4e55-a3bc-984c152c5ee2-kube-api-access-8px2r\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.773026 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.772923    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ae31de6-e744-4e9a-a9dd-7c5203acad17-crio-socket\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.773026 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.772969    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-service-ca-bundle\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.773026 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.773004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5f14faa0-564b-4e55-a3bc-984c152c5ee2-archives-path\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.773224 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.773054    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-trusted-ca-bundle\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.773224 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.773081    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwfkd\" (UniqueName: \"kubernetes.io/projected/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-api-access-vwfkd\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.773487 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.773460    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ae31de6-e744-4e9a-a9dd-7c5203acad17-crio-socket\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.773664 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.773644    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5f14faa0-564b-4e55-a3bc-984c152c5ee2-archives-path\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.773780 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.773766    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ae31de6-e744-4e9a-a9dd-7c5203acad17-data-volume\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.774257 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.773853    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.774257 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.773896    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.774257 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:33.774010    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 03 19:26:33.774257 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:33.774140    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:34.274120712 +0000 UTC m=+36.208658489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : configmap "kube-rbac-proxy" not found
Nov 03 19:26:33.774257 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.774165    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ae31de6-e744-4e9a-a9dd-7c5203acad17-data-volume\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.774257 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:33.774215    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:33.774257 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:33.774243    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:34.274235034 +0000 UTC m=+36.208772799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:33.774557 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.774377    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-trusted-ca-bundle\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.774953 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.774929    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-service-ca-bundle\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.782533 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.782508    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr"]
Nov 03 19:26:33.782653 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.782637    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct"
Nov 03 19:26:33.786812 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.786643    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct"]
Nov 03 19:26:33.786812 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.786734    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 03 19:26:33.786812 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.786774    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:33.787013 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.786817    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gh7dx\""
Nov 03 19:26:33.795907 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.795306    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8px2r\" (UniqueName: \"kubernetes.io/projected/5f14faa0-564b-4e55-a3bc-984c152c5ee2-kube-api-access-8px2r\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.796146 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.796125    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwfkd\" (UniqueName: \"kubernetes.io/projected/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-api-access-vwfkd\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:33.835392 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.835373    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/1.log"
Nov 03 19:26:33.835985 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.835969    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/0.log"
Nov 03 19:26:33.836073 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.835998    2574 generic.go:356] "Generic (PLEG): container finished" podID="c5f64340-0f40-47b9-949c-6f25f4ef05ab" containerID="63aef6de7c7db6a316d45c39164997b7650203900a9a55fc42d2ff324c7e91f1" exitCode=255
Nov 03 19:26:33.836118 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.836072    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6" event={"ID":"c5f64340-0f40-47b9-949c-6f25f4ef05ab","Type":"ContainerDied","Data":"63aef6de7c7db6a316d45c39164997b7650203900a9a55fc42d2ff324c7e91f1"}
Nov 03 19:26:33.836118 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.836109    2574 scope.go:122] "RemoveContainer" containerID="282d3f9ada3abd8f83b497bfa6fb2498e98ebb9772821316ffca08c796b41261"
Nov 03 19:26:33.836530 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.836397    2574 scope.go:122] "RemoveContainer" containerID="63aef6de7c7db6a316d45c39164997b7650203900a9a55fc42d2ff324c7e91f1"
Nov 03 19:26:33.836612 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:33.836590    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-hb5c6_openshift-console-operator(c5f64340-0f40-47b9-949c-6f25f4ef05ab)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6" podUID="c5f64340-0f40-47b9-949c-6f25f4ef05ab"
Nov 03 19:26:33.838579 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.838558    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2l9ws" event={"ID":"f21610fc-0c1e-4ba3-b54e-d3db441d629f","Type":"ContainerStarted","Data":"1fb59f3014e2bb210aef2057069c03ed78c84a6822166269be221c2393a8b8d3"}
Nov 03 19:26:33.841898 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.841503    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g9g77" event={"ID":"4d145360-58fb-4376-8126-7933a7abaae2","Type":"ContainerStarted","Data":"57b2d22a59cbd7fbd7d6912307f7d13b2efc1cb330072849be3d374ef0b1ad80"}
Nov 03 19:26:33.844269 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.844254    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-2qjr4" event={"ID":"c4f3fa56-4b85-46bd-8c97-94da036ce8bb","Type":"ContainerStarted","Data":"b85ca642dfb8149c2ee9c07399425065ee7f6b539f46f18fff314ff5e0a60a20"}
Nov 03 19:26:33.846087 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.846067    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25" event={"ID":"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd","Type":"ContainerStarted","Data":"ea926e5fc4b3c893162b3bab6524d22795d748dd1a52f7d715b7f264917f1146"}
Nov 03 19:26:33.848160 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.848139    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7" event={"ID":"ced54a02-fff8-43e5-bebd-80a41ed74f0b","Type":"ContainerStarted","Data":"a08be9206fa32f2deef1cf8ea3b8a2a23e6aeb1045b66b383024b230032d1061"}
Nov 03 19:26:33.849398 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.849348    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txdf4" event={"ID":"074a9efa-d51e-4192-841c-3cd308d89945","Type":"ContainerStarted","Data":"30b321853ecbc0d3956bb74999bd7abac312a1e68c7d3acd6a1507b488e0907b"}
Nov 03 19:26:33.853365 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.853350    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj" event={"ID":"d3f48c1f-1aad-49d7-9063-f6cf60c43907","Type":"ContainerStarted","Data":"af2e9ed625e0e34dd77f82b08c80935a4c489dce53c3ee550627050d26e6d839"}
Nov 03 19:26:33.860133 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.859317    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-fjgtn"]
Nov 03 19:26:33.871880 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.871533    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:33.871880 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.871549    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-fjgtn"]
Nov 03 19:26:33.874555 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.874510    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpqdq\" (UniqueName: \"kubernetes.io/projected/3d9a6763-8bdf-479b-abae-a86cf51bb992-kube-api-access-zpqdq\") pod \"migrator-66df5d496b-vnqjr\" (UID: \"3d9a6763-8bdf-479b-abae-a86cf51bb992\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr"
Nov 03 19:26:33.885282 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.884450    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj" podStartSLOduration=323.235867093 podStartE2EDuration="5m27.884437971s" totalImagesPullingTime="4.648570878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:21:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:33.882824017 +0000 UTC m=+35.817361806" watchObservedRunningTime="2026-11-03 19:26:33.884437971 +0000 UTC m=+35.818975752"
Nov 03 19:26:33.885888 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.885866    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 03 19:26:33.885980 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.885882    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 03 19:26:33.887808 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.887790    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 03 19:26:33.892302 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.892281    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 03 19:26:33.893755 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.893735    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qmjpj\""
Nov 03 19:26:33.909528 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.909483    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7" podStartSLOduration=316.459091611 podStartE2EDuration="5m21.909468065s" totalImagesPullingTime="5.450376454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:33.908029192 +0000 UTC m=+35.842566980" watchObservedRunningTime="2026-11-03 19:26:33.909468065 +0000 UTC m=+35.844005854"
Nov 03 19:26:33.930471 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.930430    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25" podStartSLOduration=323.240024733 podStartE2EDuration="5m27.93041947s" totalImagesPullingTime="4.690394737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:21:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:33.929448402 +0000 UTC m=+35.863986191" watchObservedRunningTime="2026-11-03 19:26:33.93041947 +0000 UTC m=+35.864957251"
Nov 03 19:26:33.933790 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.933770    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:33.956964 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.956912    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-2qjr4" podStartSLOduration=323.30451487 podStartE2EDuration="5m27.956899125s" totalImagesPullingTime="4.652384255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:21:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:33.956492353 +0000 UTC m=+35.891030163" watchObservedRunningTime="2026-11-03 19:26:33.956899125 +0000 UTC m=+35.891436913"
Nov 03 19:26:33.975755 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.975726    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-key\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:33.975981 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.975906    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-cabundle\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:33.975981 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.975971    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6kwf\" (UniqueName: \"kubernetes.io/projected/71dce3c0-a252-4070-906d-451755373263-kube-api-access-b6kwf\") pod \"migrator-66df5d496b-wptct\" (UID: \"71dce3c0-a252-4070-906d-451755373263\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct"
Nov 03 19:26:33.976275 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.976255    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqdq\" (UniqueName: \"kubernetes.io/projected/3d9a6763-8bdf-479b-abae-a86cf51bb992-kube-api-access-zpqdq\") pod \"migrator-66df5d496b-vnqjr\" (UID: \"3d9a6763-8bdf-479b-abae-a86cf51bb992\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr"
Nov 03 19:26:33.976366 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.976349    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwm4z\" (UniqueName: \"kubernetes.io/projected/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-kube-api-access-nwm4z\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:33.979593 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.979017    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g9g77" podStartSLOduration=2.023880309 podStartE2EDuration="6.979004574s" totalImagesPullingTime="4.955124265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:33.977256969 +0000 UTC m=+35.911794757" watchObservedRunningTime="2026-11-03 19:26:33.979004574 +0000 UTC m=+35.913542360"
Nov 03 19:26:33.994551 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:33.994524    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpqdq\" (UniqueName: \"kubernetes.io/projected/3d9a6763-8bdf-479b-abae-a86cf51bb992-kube-api-access-zpqdq\") pod \"migrator-66df5d496b-vnqjr\" (UID: \"3d9a6763-8bdf-479b-abae-a86cf51bb992\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr"
Nov 03 19:26:34.077138 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.077106    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-key\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:34.077290 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.077175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-cabundle\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:34.077290 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.077208    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6kwf\" (UniqueName: \"kubernetes.io/projected/71dce3c0-a252-4070-906d-451755373263-kube-api-access-b6kwf\") pod \"migrator-66df5d496b-wptct\" (UID: \"71dce3c0-a252-4070-906d-451755373263\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct"
Nov 03 19:26:34.077410 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.077329    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwm4z\" (UniqueName: \"kubernetes.io/projected/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-kube-api-access-nwm4z\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:34.078226 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.078171    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-cabundle\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:34.080209 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.080162    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-key\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:34.090854 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.090830    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6kwf\" (UniqueName: \"kubernetes.io/projected/71dce3c0-a252-4070-906d-451755373263-kube-api-access-b6kwf\") pod \"migrator-66df5d496b-wptct\" (UID: \"71dce3c0-a252-4070-906d-451755373263\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct"
Nov 03 19:26:34.091137 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.091111    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwm4z\" (UniqueName: \"kubernetes.io/projected/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-kube-api-access-nwm4z\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:34.092730 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.092710    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr"
Nov 03 19:26:34.101001 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.100980    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct"
Nov 03 19:26:34.191857 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.191824    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:34.260863 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.260780    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-h9gh8-2nbhw"]
Nov 03 19:26:34.263257 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:34.263226    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f14faa0_564b_4e55_a3bc_984c152c5ee2.slice/crio-fb2426a2659140ca1740b13b8a78e6e3f5ec409087f31c8d795ff2452e1f9c59 WatchSource:0}: Error finding container fb2426a2659140ca1740b13b8a78e6e3f5ec409087f31c8d795ff2452e1f9c59: Status 404 returned error can't find the container with id fb2426a2659140ca1740b13b8a78e6e3f5ec409087f31c8d795ff2452e1f9c59
Nov 03 19:26:34.280358 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.280298    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:34.280358 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.280349    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:34.280595 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:34.280576    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 03 19:26:34.280647 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:34.280638    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:35.280620763 +0000 UTC m=+37.215158542 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : configmap "kube-rbac-proxy" not found
Nov 03 19:26:34.281066 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:34.281030    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:34.281147 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:34.281102    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:35.281086458 +0000 UTC m=+37.215624239 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:34.409221 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.409196    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct"]
Nov 03 19:26:34.410289 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:34.410258    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71dce3c0_a252_4070_906d_451755373263.slice/crio-99fa95ce1a0381fe9d971595b48c44e131789da461af47cd9b6fe7d5b6f1518e WatchSource:0}: Error finding container 99fa95ce1a0381fe9d971595b48c44e131789da461af47cd9b6fe7d5b6f1518e: Status 404 returned error can't find the container with id 99fa95ce1a0381fe9d971595b48c44e131789da461af47cd9b6fe7d5b6f1518e
Nov 03 19:26:34.417729 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.417703    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr"]
Nov 03 19:26:34.418386 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:34.418363    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d9a6763_8bdf_479b_abae_a86cf51bb992.slice/crio-347e39e799415f1ae8e4989ef9a6f43d5e2630a5c377d6ec0c4e94f1cad7bcdd WatchSource:0}: Error finding container 347e39e799415f1ae8e4989ef9a6f43d5e2630a5c377d6ec0c4e94f1cad7bcdd: Status 404 returned error can't find the container with id 347e39e799415f1ae8e4989ef9a6f43d5e2630a5c377d6ec0c4e94f1cad7bcdd
Nov 03 19:26:34.502943 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.502914    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-fjgtn"]
Nov 03 19:26:34.505435 ip-10-0-143-212 kubenswrapper[2574]: W1103 19:26:34.505407    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a3b49cb_fa12_4a54_8eb7_bd7b9d14bb14.slice/crio-7d6e3018dea914dc3c6cb2ab7ba7835e80f68f16c53e41bafde0dd2a8c0eb182 WatchSource:0}: Error finding container 7d6e3018dea914dc3c6cb2ab7ba7835e80f68f16c53e41bafde0dd2a8c0eb182: Status 404 returned error can't find the container with id 7d6e3018dea914dc3c6cb2ab7ba7835e80f68f16c53e41bafde0dd2a8c0eb182
Nov 03 19:26:34.858576 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.858507    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/1.log"
Nov 03 19:26:34.859008 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.858875    2574 scope.go:122] "RemoveContainer" containerID="63aef6de7c7db6a316d45c39164997b7650203900a9a55fc42d2ff324c7e91f1"
Nov 03 19:26:34.859122 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:34.859093    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-hb5c6_openshift-console-operator(c5f64340-0f40-47b9-949c-6f25f4ef05ab)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6" podUID="c5f64340-0f40-47b9-949c-6f25f4ef05ab"
Nov 03 19:26:34.860905 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.860877    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2l9ws" event={"ID":"f21610fc-0c1e-4ba3-b54e-d3db441d629f","Type":"ContainerStarted","Data":"708f782d4058ddcf108bb69ef3a1c8f2698732084b1ed7b2758a4a7e71841702"}
Nov 03 19:26:34.860998 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.860940    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:34.862793 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.862768    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn" event={"ID":"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14","Type":"ContainerStarted","Data":"6e82fb938d54ee6b97874192f9542fabf4a5e0106f93c413349266beedbdc433"}
Nov 03 19:26:34.862867 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.862809    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn" event={"ID":"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14","Type":"ContainerStarted","Data":"7d6e3018dea914dc3c6cb2ab7ba7835e80f68f16c53e41bafde0dd2a8c0eb182"}
Nov 03 19:26:34.864235 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.864211    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct" event={"ID":"71dce3c0-a252-4070-906d-451755373263","Type":"ContainerStarted","Data":"99fa95ce1a0381fe9d971595b48c44e131789da461af47cd9b6fe7d5b6f1518e"}
Nov 03 19:26:34.865483 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.865457    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr" event={"ID":"3d9a6763-8bdf-479b-abae-a86cf51bb992","Type":"ContainerStarted","Data":"347e39e799415f1ae8e4989ef9a6f43d5e2630a5c377d6ec0c4e94f1cad7bcdd"}
Nov 03 19:26:34.867193 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.867164    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-h9gh8-2nbhw" event={"ID":"5f14faa0-564b-4e55-a3bc-984c152c5ee2","Type":"ContainerStarted","Data":"ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27"}
Nov 03 19:26:34.867280 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.867202    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-h9gh8-2nbhw" event={"ID":"5f14faa0-564b-4e55-a3bc-984c152c5ee2","Type":"ContainerStarted","Data":"fb2426a2659140ca1740b13b8a78e6e3f5ec409087f31c8d795ff2452e1f9c59"}
Nov 03 19:26:34.908340 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.908295    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2l9ws" podStartSLOduration=35.287546372 podStartE2EDuration="35.908282796s" totalImagesPullingTime="620.736424ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:34.907418231 +0000 UTC m=+36.841956019" watchObservedRunningTime="2026-11-03 19:26:34.908282796 +0000 UTC m=+36.842820583"
Nov 03 19:26:34.930505 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.928944    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-h9gh8-2nbhw" podStartSLOduration=1.9287526339999999 podStartE2EDuration="1.928752634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:26:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:34.928196752 +0000 UTC m=+36.862734540" watchObservedRunningTime="2026-11-03 19:26:34.928752634 +0000 UTC m=+36.863290422"
Nov 03 19:26:34.952356 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:34.951908    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn" podStartSLOduration=1.95189626 podStartE2EDuration="1.95189626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:26:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:34.949904574 +0000 UTC m=+36.884442362" watchObservedRunningTime="2026-11-03 19:26:34.95189626 +0000 UTC m=+36.886434044"
Nov 03 19:26:35.294863 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:35.294828    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:35.295019 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:35.294876    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:35.295019 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:35.294906    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:35.295019 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:35.294928    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:35.295019 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:35.294956    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:35.295019 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:35.294983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:35.295019 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:35.294999    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:35.295019 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:35.295020    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:35.295054    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:35.295082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295230    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295234    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295262    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle podName:b638dbdf-a1bf-4271-be87-4937d20d4a61 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:43.295241491 +0000 UTC m=+45.229779272 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle") pod "router-default-687ff884df-rq9lk" (UID: "b638dbdf-a1bf-4271-be87-4937d20d4a61") : configmap references non-existent config key: service-ca.crt
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295266    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295289    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r: secret "image-registry-tls" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295294    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls podName:10683ad8-f14a-4034-9768-bb8b168141d1 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:43.2952776 +0000 UTC m=+45.229815381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls") pod "dns-default-nlt2r" (UID: "10683ad8-f14a-4034-9768-bb8b168141d1") : secret "dns-default-metrics-tls" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295313    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls podName:68217d54-7abd-4c3f-8aef-3dc2979c1014 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:43.295302641 +0000 UTC m=+45.229840417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6gdg5" (UID: "68217d54-7abd-4c3f-8aef-3dc2979c1014") : secret "samples-operator-tls" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295321    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295329    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls podName:ded5b685-660b-4d0f-8100-9c364ab50c7f nodeName:}" failed. No retries permitted until 2026-11-03 19:26:43.295320722 +0000 UTC m=+45.229858491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls") pod "image-registry-7cc7bcdcc7-r2w7r" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f") : secret "image-registry-tls" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295315    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295352    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295357    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:37.295343901 +0000 UTC m=+39.229881678 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295366    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 19:26:35.295367 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295374    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert podName:51919e3c-99a2-484b-96ac-35948a9eec74 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:43.295365549 +0000 UTC m=+45.229903327 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pv5k4" (UID: "51919e3c-99a2-484b-96ac-35948a9eec74") : secret "networking-console-plugin-cert" not found
Nov 03 19:26:35.295981 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295383    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d96b8f96d-xh9sn: secret "image-registry-tls" not found
Nov 03 19:26:35.295981 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295391    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls podName:13fdaf11-64d1-445d-b055-fbec6b90c1bb nodeName:}" failed. No retries permitted until 2026-11-03 19:26:43.295381519 +0000 UTC m=+45.229919286 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-x8xpg" (UID: "13fdaf11-64d1-445d-b055-fbec6b90c1bb") : secret "cluster-monitoring-operator-tls" not found
Nov 03 19:26:35.295981 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295409    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 03 19:26:35.295981 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295419    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls podName:d4a695e5-8735-4bad-8903-13c26e5bf74a nodeName:}" failed. No retries permitted until 2026-11-03 19:26:43.295408834 +0000 UTC m=+45.229946613 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls") pod "image-registry-7d96b8f96d-xh9sn" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a") : secret "image-registry-tls" not found
Nov 03 19:26:35.295981 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295431    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 03 19:26:35.295981 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295439    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs podName:b638dbdf-a1bf-4271-be87-4937d20d4a61 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:43.295429467 +0000 UTC m=+45.229967272 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs") pod "router-default-687ff884df-rq9lk" (UID: "b638dbdf-a1bf-4271-be87-4937d20d4a61") : secret "router-metrics-certs-default" not found
Nov 03 19:26:35.295981 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:35.295461    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:37.295452532 +0000 UTC m=+39.229990300 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : configmap "kube-rbac-proxy" not found
Nov 03 19:26:36.829246 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:36.829222    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rbszw_61c81f40-4aae-4c97-9be9-30da0effd8cb/dns-node-resolver/0.log"
Nov 03 19:26:36.882633 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:36.882603    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr" event={"ID":"3d9a6763-8bdf-479b-abae-a86cf51bb992","Type":"ContainerStarted","Data":"23c8cc4d948b77a92f09bf736da493ad42ea9d30a9efe0021306aa0f65785c93"}
Nov 03 19:26:36.884363 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:36.884329    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct" event={"ID":"71dce3c0-a252-4070-906d-451755373263","Type":"ContainerStarted","Data":"c664b2fe8b0fc0f491de294fb3727152bae7e155dd30b65b4abe93bc1edadd88"}
Nov 03 19:26:36.903711 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:36.903661    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr" podStartSLOduration=2.045857342 podStartE2EDuration="3.90364925s" totalImagesPullingTime="1.857791908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:26:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:36.90309134 +0000 UTC m=+38.837629128" watchObservedRunningTime="2026-11-03 19:26:36.90364925 +0000 UTC m=+38.838187038"
Nov 03 19:26:36.925563 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:36.925515    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct" podStartSLOduration=2.06604912 podStartE2EDuration="3.925502932s" totalImagesPullingTime="1.859453812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:26:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:36.924478209 +0000 UTC m=+38.859015998" watchObservedRunningTime="2026-11-03 19:26:36.925502932 +0000 UTC m=+38.860040718"
Nov 03 19:26:37.318725 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:37.318687    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:37.318915 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:37.318740    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:37.318915 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:37.318867    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:37.318915 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:37.318890    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 03 19:26:37.319103 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:37.318936    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:41.318917273 +0000 UTC m=+43.253455037 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:37.319103 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:37.318951    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:41.318945201 +0000 UTC m=+43.253482966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : configmap "kube-rbac-proxy" not found
Nov 03 19:26:37.829444 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:37.829414    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ss5gj_ca7d9994-2fc4-4328-9c50-918954121ead/node-ca/0.log"
Nov 03 19:26:37.844559 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:37.844529    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:37.844708 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:37.844578    2574 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:37.844967 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:37.844952    2574 scope.go:122] "RemoveContainer" containerID="63aef6de7c7db6a316d45c39164997b7650203900a9a55fc42d2ff324c7e91f1"
Nov 03 19:26:37.845200 ip-10-0-143-212 kubenswrapper[2574]: E1103 19:26:37.845170    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-hb5c6_openshift-console-operator(c5f64340-0f40-47b9-949c-6f25f4ef05ab)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6" podUID="c5f64340-0f40-47b9-949c-6f25f4ef05ab"
Nov 03 19:26:38.229869 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:38.229829    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g9g77_4d145360-58fb-4376-8126-7933a7abaae2/serve-healthcheck-canary/0.log"
Nov 03 19:26:38.430477 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:38.430451    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-vnqjr_3d9a6763-8bdf-479b-abae-a86cf51bb992/migrator/0.log"
Nov 03 19:26:38.630255 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:38.630190    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-wptct_71dce3c0-a252-4070-906d-451755373263/migrator/0.log"
Nov 03 19:26:38.836268 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:38.836241    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-dpshj_d3f48c1f-1aad-49d7-9063-f6cf60c43907/kube-storage-version-migrator-operator/0.log"
Nov 03 19:26:38.892373 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:38.892297    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txdf4" event={"ID":"074a9efa-d51e-4192-841c-3cd308d89945","Type":"ContainerStarted","Data":"924fea46aba428c70ff985e86cd39bb2963283ecd0aa055f4552882794f77d27"}
Nov 03 19:26:38.916475 ip-10-0-143-212 kubenswrapper[2574]: I1103 19:26:38.916423    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-txdf4" podStartSLOduration=34.406518844 podStartE2EDuration="39.916410843s" totalImagesPullingTime="5.509891999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 19:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:38.913192581 +0000 UTC m=+40.847730367" watchObservedRunningTime="2026-11-03 19:26:38.916410843 +0000 UTC m=+40.850948670"
Nov 03 19:26:39.012897 ip-10-0-143-212 systemd[1]: Stopping Kubernetes Kubelet...
Nov 03 19:26:39.032468 ip-10-0-143-212 systemd[1]: kubelet.service: Deactivated successfully.
Nov 03 19:26:39.032726 ip-10-0-143-212 systemd[1]: Stopped Kubernetes Kubelet.
Nov 03 19:26:39.032985 ip-10-0-143-212 systemd[1]: kubelet.service: Consumed 3.167s CPU time.
Nov 03 19:26:39.048628 ip-10-0-143-212 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 19:26:39.144214 ip-10-0-143-212 kubenswrapper[5051]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 19:26:39.144214 ip-10-0-143-212 kubenswrapper[5051]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 19:26:39.144214 ip-10-0-143-212 kubenswrapper[5051]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 19:26:39.144214 ip-10-0-143-212 kubenswrapper[5051]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 19:26:39.146244 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146164    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:26:39.146244 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146243    5051 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:26:39.146244 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146247    5051 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146251    5051 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146255    5051 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146259    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146262    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146267    5051 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146272    5051 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146275    5051 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146278    5051 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146281    5051 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146284    5051 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146286    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146289    5051 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146291    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146294    5051 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146297    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146303    5051 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146306    5051 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146309    5051 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:26:39.146346 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146311    5051 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146315    5051 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146318    5051 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146320    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146323    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146326    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146328    5051 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146331    5051 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146334    5051 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146337    5051 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146340    5051 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146343    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146345    5051 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146348    5051 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146351    5051 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146353    5051 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146357    5051 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146359    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146362    5051 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146365    5051 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:26:39.146837 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146367    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146370    5051 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146373    5051 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146376    5051 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146378    5051 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146381    5051 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146383    5051 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146386    5051 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146389    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146392    5051 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146394    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146398    5051 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146402    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146405    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146408    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146411    5051 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146413    5051 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146416    5051 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146419    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:26:39.147351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146421    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146425    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146428    5051 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146431    5051 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146434    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146437    5051 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146440    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146443    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146445    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146448    5051 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146450    5051 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146453    5051 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146456    5051 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146458    5051 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146461    5051 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146464    5051 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146467    5051 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146469    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146472    5051 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:26:39.147844 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146475    5051 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146478    5051 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146480    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146483    5051 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146493    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146496    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146499    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146502    5051 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146505    5051 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146508    5051 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146511    5051 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146513    5051 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146516    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146518    5051 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146521    5051 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146524    5051 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146526    5051 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146529    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146531    5051 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146533    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:26:39.148361 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146537    5051 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146540    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146542    5051 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146544    5051 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146547    5051 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146549    5051 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.146555    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147296    5051 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147305    5051 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147308    5051 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147312    5051 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147315    5051 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147318    5051 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147321    5051 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147324    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147327    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147329    5051 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147332    5051 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147335    5051 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147337    5051 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:26:39.148893 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147340    5051 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147343    5051 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147346    5051 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147348    5051 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147351    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147353    5051 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147356    5051 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147359    5051 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147361    5051 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147364    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147367    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147370    5051 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147373    5051 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147375    5051 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147378    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147380    5051 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147383    5051 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147385    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147388    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147391    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:26:39.149444 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147394    5051 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147396    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147398    5051 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147401    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147404    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147407    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147410    5051 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147412    5051 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147415    5051 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147417    5051 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147420    5051 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147422    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147425    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147428    5051 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147430    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147433    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147435    5051 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147440    5051 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147443    5051 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:26:39.149962 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147446    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147450    5051 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147452    5051 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147455    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147457    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147460    5051 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147462    5051 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147465    5051 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147467    5051 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147470    5051 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147472    5051 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147475    5051 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147477    5051 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147480    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147483    5051 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147485    5051 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147489    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147491    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147494    5051 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:26:39.150469 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147497    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147499    5051 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147502    5051 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147504    5051 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147506    5051 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147509    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147512    5051 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147515    5051 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147517    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147520    5051 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147522    5051 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147525    5051 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147527    5051 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147530    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147532    5051 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147535    5051 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147537    5051 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147540    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147542    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147545    5051 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:26:39.150984 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147547    5051 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147550    5051 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147552    5051 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147555    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147557    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147561    5051 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147564    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147568    5051 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147571    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147573    5051 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147576    5051 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147578    5051 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147581    5051 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147584    5051 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.147587    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148071    5051 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148077    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148081    5051 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148084    5051 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:26:39.151517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148087    5051 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148089    5051 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148092    5051 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148094    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148097    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148100    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148102    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148105    5051 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148107    5051 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148110    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148112    5051 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148115    5051 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148117    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148119    5051 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148122    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148125    5051 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148128    5051 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148130    5051 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148133    5051 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:26:39.152034 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148136    5051 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148139    5051 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148141    5051 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148145    5051 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148147    5051 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148150    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148153    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148155    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148158    5051 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148160    5051 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148163    5051 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148165    5051 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148168    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148170    5051 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148173    5051 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148175    5051 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148178    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148180    5051 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148183    5051 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148185    5051 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:26:39.152523 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148188    5051 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148190    5051 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148193    5051 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148196    5051 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148199    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148201    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148204    5051 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148206    5051 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148209    5051 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148212    5051 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148214    5051 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148217    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148219    5051 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148222    5051 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148225    5051 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148228    5051 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148230    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148233    5051 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148236    5051 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148238    5051 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:26:39.153019 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148241    5051 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148244    5051 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148246    5051 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148249    5051 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148251    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148256    5051 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148261    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148264    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148267    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148269    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148272    5051 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148274    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148277    5051 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148280    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148282    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148284    5051 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148287    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148289    5051 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148292    5051 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:26:39.153525 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148294    5051 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148297    5051 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148300    5051 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148302    5051 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148305    5051 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148307    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148310    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148312    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148315    5051 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148318    5051 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148321    5051 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148323    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148326    5051 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148328    5051 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148331    5051 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148334    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148336    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148339    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148342    5051 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:26:39.154029 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148346    5051 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148350    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148354    5051 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148357    5051 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.148359    5051 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148453    5051 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148461    5051 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148467    5051 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148473    5051 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148480    5051 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148485    5051 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148492    5051 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148498    5051 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148501    5051 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148504    5051 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148511    5051 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148514    5051 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148518    5051 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148521    5051 flags.go:64] FLAG: --cgroup-root=""
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148524    5051 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148527    5051 flags.go:64] FLAG: --client-ca-file=""
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148530    5051 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148538    5051 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148543    5051 flags.go:64] FLAG: --cluster-domain=""
Nov 03 19:26:39.154804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148546    5051 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148550    5051 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148553    5051 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148556    5051 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148566    5051 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148569    5051 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148573    5051 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148576    5051 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148579    5051 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148582    5051 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148585    5051 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148589    5051 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148591    5051 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148596    5051 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148600    5051 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148603    5051 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148606    5051 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148610    5051 flags.go:64] FLAG: --enable-server="true"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148613    5051 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148617    5051 flags.go:64] FLAG: --event-burst="100"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148620    5051 flags.go:64] FLAG: --event-qps="50"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148623    5051 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148626    5051 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148630    5051 flags.go:64] FLAG: --eviction-hard=""
Nov 03 19:26:39.155507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148634    5051 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148637    5051 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148640    5051 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148644    5051 flags.go:64] FLAG: --eviction-soft=""
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148647    5051 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148650    5051 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148653    5051 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148657    5051 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148660    5051 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148663    5051 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148666    5051 flags.go:64] FLAG: --feature-gates=""
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148670    5051 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148673    5051 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148676    5051 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148679    5051 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148682    5051 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148685    5051 flags.go:64] FLAG: --help="false"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148689    5051 flags.go:64] FLAG: --hostname-override="ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148692    5051 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148695    5051 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148698    5051 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148701    5051 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148706    5051 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 19:26:39.156150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148709    5051 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148712    5051 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148715    5051 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148718    5051 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148721    5051 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148724    5051 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148727    5051 flags.go:64] FLAG: --kube-reserved=""
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148730    5051 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148735    5051 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148738    5051 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148741    5051 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148744    5051 flags.go:64] FLAG: --lock-file=""
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148747    5051 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148750    5051 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148753    5051 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148758    5051 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148761    5051 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148765    5051 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148768    5051 flags.go:64] FLAG: --logging-format="text"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148771    5051 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148774    5051 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148777    5051 flags.go:64] FLAG: --manifest-url=""
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148780    5051 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148784    5051 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148787    5051 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 19:26:39.156792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148791    5051 flags.go:64] FLAG: --max-pods="110"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148794    5051 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148797    5051 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148800    5051 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148803    5051 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148806    5051 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148810    5051 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148813    5051 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148820    5051 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148823    5051 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148826    5051 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148829    5051 flags.go:64] FLAG: --pod-cidr=""
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148832    5051 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148835    5051 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148838    5051 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148845    5051 flags.go:64] FLAG: --port="10250"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148848    5051 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148851    5051 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0a7e2189fb8b0b189"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148854    5051 flags.go:64] FLAG: --qos-reserved=""
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148857    5051 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148861    5051 flags.go:64] FLAG: --register-node="true"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148864    5051 flags.go:64] FLAG: --register-with-taints=""
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148868    5051 flags.go:64] FLAG: --registry-burst="10"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148871    5051 flags.go:64] FLAG: --registry-qps="5"
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148874    5051 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148877    5051 flags.go:64] FLAG: --reserved-memory=""
Nov 03 19:26:39.157458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148884    5051 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148887    5051 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148890    5051 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148893    5051 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148896    5051 flags.go:64] FLAG: --runonce="false"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148899    5051 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148903    5051 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148906    5051 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148909    5051 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148912    5051 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148915    5051 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148918    5051 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148923    5051 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148926    5051 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148929    5051 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148932    5051 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148935    5051 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148938    5051 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148941    5051 flags.go:64] FLAG: --system-cgroups=""
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148944    5051 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148950    5051 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148954    5051 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148958    5051 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148962    5051 flags.go:64] FLAG: --tls-min-version=""
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148965    5051 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 19:26:39.158183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148968    5051 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148972    5051 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148975    5051 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148978    5051 flags.go:64] FLAG: --v="2"
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148982    5051 flags.go:64] FLAG: --version="false"
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148986    5051 flags.go:64] FLAG: --vmodule=""
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148990    5051 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.148994    5051 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149138    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149142    5051 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149145    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149148    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149151    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149153    5051 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149156    5051 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149159    5051 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149161    5051 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149164    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149167    5051 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149172    5051 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149174    5051 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149177    5051 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:26:39.159180 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149180    5051 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149182    5051 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149185    5051 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149188    5051 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149191    5051 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149193    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149197    5051 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149200    5051 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149202    5051 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149205    5051 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149208    5051 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149210    5051 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149213    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149215    5051 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149218    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149221    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149223    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149226    5051 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149228    5051 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:26:39.159797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149231    5051 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149234    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149236    5051 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149240    5051 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149244    5051 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149248    5051 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149250    5051 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149253    5051 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149255    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149258    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149262    5051 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149264    5051 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149267    5051 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149270    5051 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149273    5051 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149275    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149278    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149281    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149283    5051 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:26:39.160331 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149287    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149290    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149292    5051 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149295    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149298    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149300    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149303    5051 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149305    5051 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149308    5051 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149311    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149313    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149316    5051 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149318    5051 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149321    5051 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149324    5051 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149327    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149329    5051 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149331    5051 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149334    5051 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:26:39.160797 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149336    5051 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149339    5051 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149341    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149344    5051 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149348    5051 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149350    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149353    5051 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149355    5051 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149358    5051 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149361    5051 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149363    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149366    5051 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149369    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149372    5051 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149375    5051 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149379    5051 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149383    5051 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149386    5051 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149389    5051 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:26:39.161467 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149391    5051 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149394    5051 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149396    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149399    5051 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149401    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149404    5051 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149406    5051 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149409    5051 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149412    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149415    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149417    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149420    5051 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149422    5051 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149425    5051 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149427    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:26:39.162035 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149430    5051 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.149456    5051 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149490    5051 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149495    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149499    5051 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149502    5051 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149505    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149507    5051 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149510    5051 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149513    5051 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149515    5051 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149519    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149522    5051 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149524    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149527    5051 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149530    5051 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:26:39.162477 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149534    5051 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149538    5051 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149541    5051 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149544    5051 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149547    5051 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149549    5051 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149552    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149555    5051 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149557    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149560    5051 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149563    5051 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149565    5051 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149568    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149570    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149573    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149576    5051 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149579    5051 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149581    5051 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149584    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:26:39.162925 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149588    5051 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149590    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149593    5051 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149596    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149598    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149601    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149604    5051 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149606    5051 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149610    5051 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149613    5051 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149615    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149618    5051 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149621    5051 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149623    5051 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149626    5051 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149628    5051 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149631    5051 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149634    5051 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149637    5051 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149639    5051 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:26:39.163410 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149642    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149645    5051 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149647    5051 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149650    5051 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149653    5051 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149655    5051 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149658    5051 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149660    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149663    5051 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149666    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149668    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149671    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149675    5051 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149677    5051 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149680    5051 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149682    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149685    5051 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149689    5051 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149693    5051 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:26:39.163898 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149696    5051 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149700    5051 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149703    5051 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149705    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149708    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149710    5051 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149713    5051 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149716    5051 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149718    5051 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149721    5051 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149723    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149727    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149730    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149732    5051 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149735    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149737    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149740    5051 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149742    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149745    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149748    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:26:39.164383 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149750    5051 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149753    5051 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149756    5051 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149758    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149761    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149765    5051 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149768    5051 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149771    5051 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149773    5051 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149776    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149778    5051 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149781    5051 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149784    5051 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.149787    5051 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.153280    5051 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.153296    5051 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.153314    5051 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.153319    5051 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153375    5051 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153380    5051 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:26:39.164868 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153383    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153386    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153389    5051 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153391    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153394    5051 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153397    5051 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153400    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153403    5051 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153405    5051 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153408    5051 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153410    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153413    5051 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153416    5051 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153418    5051 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153421    5051 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153424    5051 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153427    5051 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153429    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153432    5051 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:26:39.165466 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153434    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153438    5051 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153440    5051 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153443    5051 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153446    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153448    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153451    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153454    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153457    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153460    5051 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153462    5051 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153465    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153468    5051 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153470    5051 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153473    5051 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153475    5051 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153478    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153482    5051 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153486    5051 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:26:39.165928 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153489    5051 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153492    5051 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153495    5051 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153498    5051 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153501    5051 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153503    5051 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153506    5051 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153509    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153512    5051 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153514    5051 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153517    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153520    5051 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153522    5051 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153525    5051 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153527    5051 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153530    5051 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153533    5051 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153536    5051 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153539    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:26:39.166400 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153542    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153544    5051 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153547    5051 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153551    5051 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153556    5051 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153558    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153561    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153564    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153566    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153569    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153571    5051 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153574    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153577    5051 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153579    5051 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153582    5051 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153585    5051 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153587    5051 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153590    5051 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153593    5051 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153595    5051 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:26:39.166883 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153598    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153600    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153603    5051 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153605    5051 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153608    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153611    5051 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153614    5051 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153616    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153619    5051 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153621    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153624    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153626    5051 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153629    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153631    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153634    5051 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153637    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153640    5051 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153643    5051 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153646    5051 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153648    5051 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:26:39.167375 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153651    5051 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153653    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153656    5051 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153658    5051 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153661    5051 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153663    5051 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153666    5051 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.153693    5051 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153825    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153830    5051 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153833    5051 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153836    5051 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153839    5051 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153842    5051 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153845    5051 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153849    5051 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:26:39.167850 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153852    5051 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153855    5051 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153857    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153860    5051 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153862    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153865    5051 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153868    5051 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153870    5051 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153873    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153876    5051 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153879    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153881    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153884    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153916    5051 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153953    5051 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153956    5051 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.153959    5051 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154003    5051 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154007    5051 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154010    5051 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:26:39.168248 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154013    5051 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154015    5051 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154018    5051 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154021    5051 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154024    5051 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154026    5051 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154029    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154036    5051 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154053    5051 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154058    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154062    5051 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154066    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154071    5051 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154076    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154080    5051 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154084    5051 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154089    5051 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154093    5051 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154098    5051 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154108    5051 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:26:39.168806 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154113    5051 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154117    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154123    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154126    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154131    5051 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154135    5051 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154141    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154146    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154154    5051 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154160    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154164    5051 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154171    5051 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154181    5051 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154187    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154192    5051 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154196    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154201    5051 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154206    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154212    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:26:39.169367 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154216    5051 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154220    5051 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154225    5051 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154233    5051 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154239    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154249    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154253    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154257    5051 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154261    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154265    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154270    5051 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154274    5051 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154279    5051 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154283    5051 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154288    5051 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154292    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154331    5051 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154338    5051 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154349    5051 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:26:39.169838 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154354    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154361    5051 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154366    5051 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154371    5051 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154375    5051 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154379    5051 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154384    5051 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154388    5051 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154392    5051 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154397    5051 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154401    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154406    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154415    5051 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154419    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154424    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154428    5051 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154432    5051 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154437    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154441    5051 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:26:39.170406 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154445    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.154512    5051 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154632    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154643    5051 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154648    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154652    5051 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154657    5051 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154662    5051 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154666    5051 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154670    5051 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154675    5051 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154679    5051 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154683    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154687    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154691    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154706    5051 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 19:26:39.171161 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154710    5051 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154714    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154718    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154723    5051 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154727    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154731    5051 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154736    5051 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154740    5051 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154744    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154748    5051 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154752    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154756    5051 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154765    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154770    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154774    5051 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154779    5051 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154783    5051 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154791    5051 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 19:26:39.171718 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154797    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154802    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154806    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154811    5051 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154815    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154820    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154829    5051 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154833    5051 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154837    5051 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154842    5051 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154846    5051 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154850    5051 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154854    5051 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154858    5051 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154863    5051 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154868    5051 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154872    5051 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154876    5051 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154880    5051 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154889    5051 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 19:26:39.172286 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154893    5051 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.154901    5051 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155121    5051 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155133    5051 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155138    5051 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155143    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155147    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155150    5051 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155154    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155157    5051 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155160    5051 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155162    5051 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155165    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155168    5051 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155170    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155173    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155175    5051 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155178    5051 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155181    5051 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 19:26:39.172760 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155183    5051 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155186    5051 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155190    5051 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155195    5051 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155198    5051 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155201    5051 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155204    5051 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155207    5051 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155211    5051 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155214    5051 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155217    5051 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155219    5051 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155222    5051 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155225    5051 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155227    5051 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155230    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155233    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155236    5051 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155239    5051 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155241    5051 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 19:26:39.173326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155246    5051 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155249    5051 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155252    5051 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155254    5051 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155258    5051 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155263    5051 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155267    5051 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155271    5051 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155275    5051 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155279    5051 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155282    5051 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155285    5051 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155287    5051 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155291    5051 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:39.155293    5051 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.155356    5051 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.156299    5051 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.156352    5051 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 03 19:26:39.173869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.156526    5051 server.go:1014] "Starting client certificate rotation"
Nov 03 19:26:39.174492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.156685    5051 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 19:26:39.174492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.156754    5051 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 19:20:58 +0000 UTC" deadline="2028-03-29 12:24:51.559873094 +0000 UTC"
Nov 03 19:26:39.174492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.156796    5051 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12280h58m12.403080427s"
Nov 03 19:26:39.174492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.156964    5051 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 19:26:39.174492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.158007    5051 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 19:26:39.174492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.159436    5051 log.go:25] "Validated CRI v1 runtime API"
Nov 03 19:26:39.174492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.161074    5051 log.go:25] "Validated CRI v1 image API"
Nov 03 19:26:39.174492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.161771    5051 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 19:26:39.174492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.163689    5051 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 95117dbc-686b-40f5-ad1d-d9492c794f91:/dev/nvme0n1p3 e2e85afc-588d-4c23-b0c8-14f5d45a6866:/dev/nvme0n1p4]
Nov 03 19:26:39.174752 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.163708    5051 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/19855996f8e578426a6a412f77cba7918b2ca057e717cdd919be47382e40f72f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/19855996f8e578426a6a412f77cba7918b2ca057e717cdd919be47382e40f72f/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1afb98175d567260d50e10f134c7996aadc6aa99934f2fb26bf7638e2609e947/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1afb98175d567260d50e10f134c7996aadc6aa99934f2fb26bf7638e2609e947/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1fb59f3014e2bb210aef2057069c03ed78c84a6822166269be221c2393a8b8d3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1fb59f3014e2bb210aef2057069c03ed78c84a6822166269be221c2393a8b8d3/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/30b321853ecbc0d3956bb74999bd7abac312a1e68c7d3acd6a1507b488e0907b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/30b321853ecbc0d3956bb74999bd7abac312a1e68c7d3acd6a1507b488e0907b/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/347e39e799415f1ae8e4989ef9a6f43d5e2630a5c377d6ec0c4e94f1cad7bcdd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/347e39e799415f1ae8e4989ef9a6f43d5e2630a5c377d6ec0c4e94f1cad7bcdd/userdata/shm major:0 minor:346 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5814ab61a89b534521e6eb380fe73e30473b71a8bd33e618c7c50d4970a0e85c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5814ab61a89b534521e6eb380fe73e30473b71a8bd33e618c7c50d4970a0e85c/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5e15edcc9f9b1866c9b9b56e137ee21d731d367d50f40ac6bc9fbc03889771ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5e15edcc9f9b1866c9b9b56e137ee21d731d367d50f40ac6bc9fbc03889771ae/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5e36bf1426511d0dba9ca46de226dbcdfab94f5468443da6ba9e8d0ccfe5ede4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5e36bf1426511d0dba9ca46de226dbcdfab94f5468443da6ba9e8d0ccfe5ede4/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6ab53da3087d52e1a1b0ef48c93ecf44e1ed66caa737f073e96aae9f70307237/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6ab53da3087d52e1a1b0ef48c93ecf44e1ed66caa737f073e96aae9f70307237/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6f37acc05f224bec8051f437f9347b9f93f9ed1b29430c7f570688d6202e7e2c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6f37acc05f224bec8051f437f9347b9f93f9ed1b29430c7f570688d6202e7e2c/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7d6e3018dea914dc3c6cb2ab7ba7835e80f68f16c53e41bafde0dd2a8c0eb182/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7d6e3018dea914dc3c6cb2ab7ba7835e80f68f16c53e41bafde0dd2a8c0eb182/userdata/shm major:0 minor:350 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/810344098a090bd63f39706afcf9d9585eb33a03429273a625d68f898c17190c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/810344098a090bd63f39706afcf9d9585eb33a03429273a625d68f898c17190c/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/89d7980e22f6af81ccdd1c7a97930400d2296be499e569c25b9a1ba7f618fe5e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/89d7980e22f6af81ccdd1c7a97930400d2296be499e569c25b9a1ba7f618fe5e/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8b2d52d95416e8b2657ace8a275373d23433fcc6b43c9ebe0d63d86c99966199/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8b2d52d95416e8b2657ace8a275373d23433fcc6b43c9ebe0d63d86c99966199/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/99fa95ce1a0381fe9d971595b48c44e131789da461af47cd9b6fe7d5b6f1518e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/99fa95ce1a0381fe9d971595b48c44e131789da461af47cd9b6fe7d5b6f1518e/userdata/shm major:0 minor:347 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9da85617cf620e26573ebd5f669f772f0a1ce4adc25ddd9c0c88bed500e72fed/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9da85617cf620e26573ebd5f669f772f0a1ce4adc25ddd9c0c88bed500e72fed/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ac6167d831859048646891f758c7383cfe54256ff8147e6c39e133a6bdd5c62a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ac6167d831859048646891f758c7383cfe54256ff8147e6c39e133a6bdd5c62a/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b220883be33b4038b1c0da7eac6ff4d8c7afee341d75efb68fc7e698bc2bd4ac/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b220883be33b4038b1c0da7eac6ff4d8c7afee341d75efb68fc7e698bc2bd4ac/userdata/shm major:0 minor:241 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b8d25f522b81416b4b2b5eca72086afb0a1674793765958b5caf41553f663e65/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b8d25f522b81416b4b2b5eca72086afb0a1674793765958b5caf41553f663e65/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c2bf2d36aec738909756a0ad609e9e57065d5b1eebfcfeba17a7ea5c518fa206/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c2bf2d36aec738909756a0ad609e9e57065d5b1eebfcfeba17a7ea5c518fa206/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c7d0befb2d6c96253f3d2eb0ce3b61b33be7c9f578898ffa084b3e50902ecfa0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c7d0befb2d6c96253f3d2eb0ce3b61b33be7c9f578898ffa084b3e50902ecfa0/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d6aebe412b486b3379cd24a3b80fa9be586a5f39e14a501de9b49ae18378f436/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d6aebe412b486b3379cd24a3b80fa9be586a5f39e14a501de9b49ae18378f436/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dc83d44b0f2991e38aa23efc9ff63fd33e647a7ac0d7090a13312cec686f34f9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dc83d44b0f2991e38aa23efc9ff63fd33e647a7ac0d7090a13312cec686f34f9/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e3c3c69d9cc25f34a9cbb0dd34b77aafb3a2a1ffd01c5569992068d6c8cf96f2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e3c3c69d9cc25f34a9cbb0dd34b77aafb3a2a1ffd01c5569992068d6c8cf96f2/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f56aa45e661ed322b53efdbdba9ccccc5bebd1b1a87ec583b79c6b6f0b911383/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f56aa45e661ed322b53efdbdba9ccccc5bebd1b1a87ec583b79c6b6f0b911383/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fb2426a2659140ca1740b13b8a78e6e3f5ec409087f31c8d795ff2452e1f9c59/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fb2426a2659140ca1740b13b8a78e6e3f5ec409087f31c8d795ff2452e1f9c59/userdata/shm major:0 minor:336 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/074a9efa-d51e-4192-841c-3cd308d89945/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/074a9efa-d51e-4192-841c-3cd308d89945/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14/volumes/kubernetes.io~projected/kube-api-access-nwm4z:{mountpoint:/var/lib/kubelet/pods/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14/volumes/kubernetes.io~projected/kube-api-access-nwm4z major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14/volumes/kubernetes.io~secret/signing-key major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10683ad8-f14a-4034-9768-bb8b168141d1/volumes/kubernetes.io~projected/kube-api-access-jvxhj:{mountpoint:/var/lib/kubelet/pods/10683ad8-f14a-4034-9768-bb8b168141d1/volumes/kubernetes.io~projected/kube-api-access-jvxhj major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13fdaf11-64d1-445d-b055-fbec6b90c1bb/volumes/kubernetes.io~projected/kube-api-access-b87d8:{mountpoint:/var/lib/kubelet/pods/13fdaf11-64d1-445d-b055-fbec6b90c1bb/volumes/kubernetes.io~projected/kube-api-access-b87d8 major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/180203ac-0a6f-4554-8ac7-aca9b55c7b3c/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/180203ac-0a6f-4554-8ac7-aca9b55c7b3c/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/180203ac-0a6f-4554-8ac7-aca9b55c7b3c/volumes/kubernetes.io~projected/kube-api-access-pjx92:{mountpoint:/var/lib/kubelet/pods/180203ac-0a6f-4554-8ac7-aca9b55c7b3c/volumes/kubernetes.io~projected/kube-api-access-pjx92 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/180203ac-0a6f-4554-8ac7-aca9b55c7b3c/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/180203ac-0a6f-4554-8ac7-aca9b55c7b3c/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ae31de6-e744-4e9a-a9dd-7c5203acad17/volumes/kubernetes.io~projected/kube-api-access-vwfkd:{mountpoint:/var/lib/kubelet/pods/3ae31de6-e744-4e9a-a9dd-7c5203acad17/volumes/kubernetes.io~projected/kube-api-access-vwfkd major:0 minor:333 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3d9a6763-8bdf-479b-abae-a86cf51bb992/volumes/kubernetes.io~projected/kube-api-access-zpqdq:{mountpoint:/var/lib/kubelet/pods/3d9a6763-8bdf-479b-abae-a86cf51bb992/volumes/kubernetes.io~projected/kube-api-access-zpqdq major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40a18e52-fbcd-4f04-a5f5-2ebafaea516a/volumes/kubernetes.io~projected/kube-api-access-x86ps:{mountpoint:/var/lib/kubelet/pods/40a18e52-fbcd-4f04-a5f5-2ebafaea516a/volumes/kubernetes.io~projected/kube-api-access-x86ps major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d145360-58fb-4376-8126-7933a7abaae2/volumes/kubernetes.io~projected/kube-api-access-f6r2k:{mountpoint:/var/lib/kubelet/pods/4d145360-58fb-4376-8126-7933a7abaae2/volumes/kubernetes.io~projected/kube-api-access-f6r2k major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d145360-58fb-4376-8126-7933a7abaae2/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/4d145360-58fb-4376-8126-7933a7abaae2/volumes/kubernetes.io~secret/cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4fd1c83c-4a87-43b1-b25e-8b9b23922aab/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/4fd1c83c-4a87-43b1-b25e-8b9b23922aab/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/563541bc-4a0f-4e6e-b6f0-ebd9264f537d/volumes/kubernetes.io~projected/kube-api-access-9p7nx:{mountpoint:/var/lib/kubelet/pods/563541bc-4a0f-4e6e-b6f0-ebd9264f537d/volumes/kubernetes.io~projected/kube-api-access-9p7nx major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5f14faa0-564b-4e55-a3bc-984c152c5ee2/volumes/kubernetes.io~projected/kube-api-access-8px2r:{mountpoint:/var/lib/kubelet/pods/5f14faa0-564b-4e55-a3bc-984c152c5ee2/volumes/kubernetes.io~projected/kube-api-access-8px2r major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61c81f40-4aae-4c97-9be9-30da0effd8cb/volumes/kubernetes.io~projected/kube-api-access-84hdd:{mountpoint:/var/lib/kubelet/pods/61c81f40-4aae-4c97-9be9-30da0effd8cb/volumes/kubernetes.io~projected/kube-api-access-84hdd major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68217d54-7abd-4c3f-8aef-3dc2979c1014/volumes/kubernetes.io~projected/kube-api-access-h2bmc:{mountpoint:/var/lib/kubelet/pods/68217d54-7abd-4c3f-8aef-3dc2979c1014/volumes/kubernetes.io~projected/kube-api-access-h2bmc major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/71dce3c0-a252-4070-906d-451755373263/volumes/kubernetes.io~projected/kube-api-access-b6kwf:{mountpoint:/var/lib/kubelet/pods/71dce3c0-a252-4070-906d-451755373263/volumes/kubernetes.io~projected/kube-api-access-b6kwf major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8529580b-2186-4766-99d3-b55374b668ef/volumes/kubernetes.io~projected/kube-api-access-j4w5n:{mountpoint:/var/lib/kubelet/pods/8529580b-2186-4766-99d3-b55374b668ef/volumes/kubernetes.io~projected/kube-api-access-j4w5n major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0/volumes/kubernetes.io~projected/kube-api-access-zlz47:{mountpoint:/var/lib/kubelet/pods/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0/volumes/kubernetes.io~projected/kube-api-access-zlz47 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8/volumes/kubernetes.io~projected/kube-api-access-nhjvn:{mountpoint:/var/lib/kubelet/pods/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8/volumes/kubernetes.io~projected/kube-api-access-nhjvn major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b638dbdf-a1bf-4271-be87-4937d20d4a61/volumes/kubernetes.io~projected/kube-api-access-grqfk:{mountpoint:/var/lib/kubelet/pods/b638dbdf-a1bf-4271-be87-4937d20d4a61/volumes/kubernetes.io~projected/kube-api-access-grqfk major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b638dbdf-a1bf-4271-be87-4937d20d4a61/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/b638dbdf-a1bf-4271-be87-4937d20d4a61/volumes/kubernetes.io~secret/default-certificate major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b638dbdf-a1bf-4271-be87-4937d20d4a61/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/b638dbdf-a1bf-4271-be87-4937d20d4a61/volumes/kubernetes.io~secret/stats-auth major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf43b3af-6048-4d1b-9a8c-96efd387cfc4/volumes/kubernetes.io~projected/kube-api-access-7fjn6:{mountpoint:/var/lib/kubelet/pods/bf43b3af-6048-4d1b-9a8c-96efd387cfc4/volumes/kubernetes.io~projected/kube-api-access-7fjn6 major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4236bfc-5fb6-40aa-8ced-a63a86f81632/volumes/kubernetes.io~projected/kube-api-access-8x5t9:{mountpoint:/var/lib/kubelet/pods/c4236bfc-5fb6-40aa-8ced-a63a86f81632/volumes/kubernetes.io~projected/kube-api-access-8x5t9 major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4f3fa56-4b85-46bd-8c97-94da036ce8bb/volumes/kubernetes.io~projected/kube-api-access-zbj7f:{mountpoint:/var/lib/kubelet/pods/c4f3fa56-4b85-46bd-8c97-94da036ce8bb/volumes/kubernetes.io~projected/kube-api-access-zbj7f major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4f3fa56-4b85-46bd-8c97-94da036ce8bb/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/c4f3fa56-4b85-46bd-8c97-94da036ce8bb/volumes/kubernetes.io~secret/serving-cert major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c5f64340-0f40-47b9-949c-6f25f4ef05ab/volumes/kubernetes.io~projected/kube-api-access-l5tfw:{mountpoint:/var/lib/kubelet/pods/c5f64340-0f40-47b9-949c-6f25f4ef05ab/volumes/kubernetes.io~projected/kube-api-access-l5tfw major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c5f64340-0f40-47b9-949c-6f25f4ef05ab/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/c5f64340-0f40-47b9-949c-6f25f4ef05ab/volumes/kubernetes.io~secret/serving-cert major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ca7d9994-2fc4-4328-9c50-918954121ead/volumes/kubernetes.io~projected/kube-api-access-m94sg:{mountpoint:/var/lib/kubelet/pods/ca7d9994-2fc4-4328-9c50-918954121ead/volumes/kubernetes.io~projected/kube-api-access-m94sg major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce9d2c60-8124-4576-835d-581115d6566f/volumes/kubernetes.io~projected/kube-api-access-2dscb:{mountpoint:/var/lib/kubelet/pods/ce9d2c60-8124-4576-835d-581115d6566f/volumes/kubernetes.io~projected/kube-api-access-2dscb major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ced54a02-fff8-43e5-bebd-80a41ed74f0b/volumes/kubernetes.io~projected/kube-api-access-rjt5p:{mountpoint:/var/lib/kubelet/pods/ced54a02-fff8-43e5-bebd-80a41ed74f0b/volumes/kubernetes.io~projected/kube-api-access-rjt5p major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfff1c16-19ba-4492-aacb-7b75bdd64c9c/volumes/kubernetes.io~projected/kube-api-access-vl2cz:{mountpoint:/var/lib/kubelet/pods/cfff1c16-19ba-4492-aacb-7b75bdd64c9c/volumes/kubernetes.io~projected/kube-api-access-vl2cz major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3f48c1f-1aad-49d7-9063-f6cf60c43907/volumes/kubernetes.io~projected/kube-api-access-v2mgz:{mountpoint:/var/lib/kubelet/pods/d3f48c1f-1aad-49d7-9063-f6cf60c43907/volumes/kubernetes.io~projected/kube-api-access-v2mgz major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3f48c1f-1aad-49d7-9063-f6cf60c43907/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/d3f48c1f-1aad-49d7-9063-f6cf60c43907/volumes/kubernetes.io~secret/serving-cert major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~projected/kube-api-access-kldqf:{mountpoint:/var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~projected/kube-api-access-kldqf major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dd1caca1-f187-4c53-860b-5443c3b95924/volumes/kubernetes.io~projected/kube-api-access-59hhh:{mountpoint:/var/lib/kubelet/pods/dd1caca1-f187-4c53-860b-5443c3b95924/volumes/kubernetes.io~projected/kube-api-access-59hhh major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~projected/kube-api-access-rfdk9:{mountpoint:/var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~projected/kube-api-access-rfdk9 major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd/volumes/kubernetes.io~projected/kube-api-access-6vdcm:{mountpoint:/var/lib/kubelet/pods/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd/volumes/kubernetes.io~projected/kube-api-access-6vdcm major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd/volumes/kubernetes.io~secret/serving-cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f21610fc-0c1e-4ba3-b54e-d3db441d629f/volumes/kubernetes.io~projected/kube-api-access-zfg6f:{mountpoint:/var/lib/kubelet/pods/f21610fc-0c1e-4ba3-b54e-d3db441d629f/volumes/kubernetes.io~projected/kube-api-access-zfg6f major:0 minor:293 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/9bd7dabf2c3f5524f8018df074cea0369f23c769080938a61225a38bc37c2e06/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/1990ab54fac1f612a4c6b581d6fdc61bd2f738c62ae66ff2943f5572ab307c43/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/d913fe9ad8dd271c6c1369b0e1283e0735ae661cbc4e321f84ee391b29a124ea/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-108:{mountpoint:/var/lib/containers/storage/overlay/447bb81c09d7fa471e1d91c5b31ef8535287759470c83797e0863fa8a44121b2/merged major:0 minor:108 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/4da1ff6a0e3321d9faacd344c515b72468cbc441b2297934027ff843a521c11d/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/ef267a09b02a74b203d0a665c7bed936204a6466a42a75e716df3923556aeaa5/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/ea1ccf19d05e73d1d71eceb79379ee30be14bbaaf62aee4a8caecf52892a681d/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/936a20d1e79857dfcf1009b051a56cedfcbbfc0243cf3bd633f91e3c49208e4c/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/d4f4883272ff144c93d35994a409520d9771ef227ddcdf05cb4e1cbb4941ea55/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/8be427f0fb974dd4ae53668568414740b99a2e76512e2e7dc8947615c7e41a0a/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/c17e588031c75ac68e290dd4aa6cfd4c87e45d7ad823e0de450b151f09c9e39e/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-145:{mountpoint:/var/lib/containers/storage/overlay/bc55f07453673152a145f63b1a0e8cb4e719b06b92191b3c2fd1c6adda5b50a0/merged major:0 minor:145 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/2c80189394be8570c2d4a35307f7eb0ad23eb7056ffa19208c055d1b86c2451a/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/b4c784f299c177dcbd09026f3f2903d4711f9e793f2493beec367353809a1dc8/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/cf79a80c5455ee05eb464f8f307b48a279d89e1bf98c7ed5f79fa4910236d64e/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/eda73a81219bf6405676bf2948d2178bd4b4c4e0151d22b7810ab70eec62f33c/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/93326461c45303b49620aa1c993a97e67365fb7dd5e33f0299afbddad99e4786/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/754d73c7ad742986c47c4682372371d38fc27d5fcd5887bd651179d08f68ee47/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/2eb1521347aa28c2294cd121f98a61d898a4e60185f7965edf29085e6183f51a/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/00265dce9562f60b3cdd1ce2f01addac271198601519d8c26e43425cd4bd0977/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/ccc5d05c40faefa1374a560cd5156583097d5e9f614b9ac3d942e557b0be30e2/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/ec94d9c7860825ef3d92ea03bd5a47b4dff5d23758a2dfc3ddc39f3853998439/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/382a21da211d9b6c7c5751c7ffa091aed680729170ed30e23342168c62079fd7/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/e14273be40b3fa38322a9925b1d93cb5dd07938f96df1e3da6defdc83d4e4b15/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/86e669c904ab137df2c0e2aef4411b7c0b7dcafa7326b7501a13bbef649929ec/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/aa288265e62922b808ece1581d2eb0d43aa872b94f9d1ace37dbd8c045eedd9d/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/b8e91ea4d0e19da421be7542243e841c549d36b0e92156715d9d05cfe11879ea/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/0b57d8ee4e7d0de887f407f6c44a4dd6c0425ab8fd54cf2b31bb8ff6c895cc57/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/6dc48e8256bee00da0c0cec2b03d6cf9f23825bfe8d917da56634bd021ae53a9/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/c6b2b62dcf4409804f8e3b9f2da3885bfba19e8bc8dd4712ed36c07eb3a7a42e/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/10f7e379cefab1396800e4fe605fa04cb1a1838a8b9f128cd6a5c1fe30c5243c/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/1c6b54f967e6dcf7be1ae1282a20b6cf7ee2974b4b978f071d61d27b2ccd9a93/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/2b1140b5a46c89c9e42a26994abf9851884d1bf5c35a27b13bd9d4ea25eebe46/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/7fe16243908f4215e642d47e776db07299d1c216fd8e7de809f302c29aa9b2fa/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/b6eb15ca1b45099983c836b027fd614d80f0129bf38a016e75ea0a8bbf0b5c6f/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/311e6050d5a904591b4e55610263f2d1cc7e4949603c2fa2344a2ee385728393/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/4c70328e3734b669e9af195c7baa6d397c4b18f6804adccb4fe5725a3fe27fa6/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/9096d66e83a37a580e4d2ed8aaf91d68945b91c40327dd5269cc83cc28aba01e/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/b0a2b78792131b16d78a97e4315b31f446fa98dfe82dccd888402fb9c35d7015/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/cbbb9af26a68ab5da5812e8ad2418278c3910a680bf7b7a3adf138b52321bcf6/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/89a269af3059125744137e044f636248687ddfc0ebe73709d6297ec908a45315/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/67688f01e4645a5949beaa0b87e31bf69c18009b26560544033d865ceb82622f/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-334:{mountpoint:/var/lib/containers/storage/overlay/c7bfb403a320e9aec675b9ecd41aec9e62f1a88c414bec4aecb1df2d5556e122/merged major:0 minor:334 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/87c043695ad09d7b5f215ae3044b891e0cb5493e58241196bf81aa442d25f5ef/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/ffafeae7be5b943a2054a5b7d14b3d621d5416f942996417a2e9c7e7cba1025e/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-356:{mountpoint:/var/lib/containers/storage/overlay/55a120faa5566317daad324cbc5517b992accd416389a970c8b39a2cd905d304/merged major:0 minor:356 fsType:overlay blockSize:0} overlay_0-362:{mountpoint:/var/lib/containers/storage/overlay/b96f53b00ec70afb9734afcd32dd55f27992bef3ef4daa82d9d34c5fb96e87ab/merged major:0 minor:362 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/c874efe836f4ce5df21a9404fbceff6e40e0159e8575c7bc2a25be9687e9cfce/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/263144dcb738583bd4f5dac838e82371a158263c415538960d3930a64804b96f/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/6242e544d9de511df2b4b3e516f0ac35f9ca0ee2d3820e0c5651ccc5430a23f7/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/016f7d492ccddc0a1eab5635422d935b64543536772cd45b42a7dca54e582575/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/18a49017c418a8525479f1205d2c3e70243936b270ca1879cc3998d57b9f47ef/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/088c667e11de0ab47199a099afbbe9be5cce466dae8f878970f7778ef0ace3fb/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/27360e07e1e45b17234de9c8367cbe42d75f43811e99c151ad3886c5de3ed454/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/c238cafb18426528bca4a481f05b6ea24a78cbed3e1ec849d5b415d63beac678/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/da5f4eb2b2cadc405522b2ccde1506febd302db2f62e9ac2e3987689d34fd307/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/7810b1b51c6cfe44f92d82345af8599146cf2b5463082e01fc0caddcd963ce65/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/fb7e8638fa310f14796e65eeefd3219a5c0d6772e30ee02f997ff739fb33c746/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/2721b65402d0ee6bb31be80430c7d37f42405ed18d3de593da41900a0fa91850/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/a2adb526c5f6612047d20f0a9140f486ee7f095ca749490524ce3baf2b72acf1/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/7086c84035f19fbcba8fd9a060177a0cc807b87f96f7ff02091fe29ee518ccf5/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/509048da5b9cf9112f5b1724f267b93ec2cf1cdb68408823b08a2a1c20f1ea99/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.172733    5051 manager.go:217] Machine: {Timestamp:2026-11-03 19:26:39.17228712 +0000 UTC m=+0.080309050 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115447 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2be45e6ee7f30ccf3893150d8d6d08 SystemUUID:ec2be45e-6ee7-f30c-cf38-93150d8d6d08 BootID:898a0088-c283-46cd-b993-4b4e8869bda5 Filesystems:[{Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/13fdaf11-64d1-445d-b055-fbec6b90c1bb/volumes/kubernetes.io~projected/kube-api-access-b87d8 DeviceMajor:0 DeviceMinor:219 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3d9a6763-8bdf-479b-abae-a86cf51bb992/volumes/kubernetes.io~projected/kube-api-access-zpqdq DeviceMajor:0 DeviceMinor:342 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/180203ac-0a6f-4554-8ac7-aca9b55c7b3c/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:64 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/40a18e52-fbcd-4f04-a5f5-2ebafaea516a/volumes/kubernetes.io~projected/kube-api-access-x86ps DeviceMajor:0 DeviceMinor:67 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:204 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b220883be33b4038b1c0da7eac6ff4d8c7afee341d75efb68fc7e698bc2bd4ac/userdata/shm DeviceMajor:0 DeviceMinor:241 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:343 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/180203ac-0a6f-4554-8ac7-aca9b55c7b3c/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/10683ad8-f14a-4034-9768-bb8b168141d1/volumes/kubernetes.io~projected/kube-api-access-jvxhj DeviceMajor:0 DeviceMinor:215 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9da85617cf620e26573ebd5f669f772f0a1ce4adc25ddd9c0c88bed500e72fed/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/810344098a090bd63f39706afcf9d9585eb33a03429273a625d68f898c17190c/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/3ae31de6-e744-4e9a-a9dd-7c5203acad17/volumes/kubernetes.io~projected/kube-api-access-vwfkd DeviceMajor:0 DeviceMinor:333 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/89d7980e22f6af81ccdd1c7a97930400d2296be499e569c25b9a1ba7f618fe5e/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:206 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~projected/kube-api-access-kldqf DeviceMajor:0 DeviceMinor:222 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b8d25f522b81416b4b2b5eca72086afb0a1674793765958b5caf41553f663e65/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-334 DeviceMajor:0 DeviceMinor:334 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ce9d2c60-8124-4576-835d-581115d6566f/volumes/kubernetes.io~projected/kube-api-access-2dscb DeviceMajor:0 DeviceMinor:59 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/30b321853ecbc0d3956bb74999bd7abac312a1e68c7d3acd6a1507b488e0907b/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:210 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e3c3c69d9cc25f34a9cbb0dd34b77aafb3a2a1ffd01c5569992068d6c8cf96f2/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d3f48c1f-1aad-49d7-9063-f6cf60c43907/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:208 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/ced54a02-fff8-43e5-bebd-80a41ed74f0b/volumes/kubernetes.io~projected/kube-api-access-rjt5p DeviceMajor:0 DeviceMinor:223 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f56aa45e661ed322b53efdbdba9ccccc5bebd1b1a87ec583b79c6b6f0b911383/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ac6167d831859048646891f758c7383cfe54256ff8147e6c39e133a6bdd5c62a/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4d145360-58fb-4376-8126-7933a7abaae2/volumes/kubernetes.io~projected/kube-api-access-f6r2k DeviceMajor:0 DeviceMinor:212 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/c5f64340-0f40-47b9-949c-6f25f4ef05ab/volumes/kubernetes.io~projected/kube-api-access-l5tfw DeviceMajor:0 DeviceMinor:224 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c4f3fa56-4b85-46bd-8c97-94da036ce8bb/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:205 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/dd1caca1-f187-4c53-860b-5443c3b95924/volumes/kubernetes.io~projected/kube-api-access-59hhh DeviceMajor:0 DeviceMinor:226 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-356 DeviceMajor:0 DeviceMinor:356 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/180203ac-0a6f-4554-8ac7-aca9b55c7b3c/volumes/kubernetes.io~projected/kube-api-access-pjx92 DeviceMajor:0 DeviceMinor:65 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5e36bf1426511d0dba9ca46de226dbcdfab94f5468443da6ba9e8d0ccfe5ede4/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14/volumes/kubernetes.io~projected/kube-api-access-nwm4z DeviceMajor:0 DeviceMinor:345 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/4fd1c83c-4a87-43b1-b25e-8b9b23922aab/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:216 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~projected/kube-api-access-rfdk9 DeviceMajor:0 DeviceMinor:218 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5f14faa0-564b-4e55-a3bc-984c152c5ee2/volumes/kubernetes.io~projected/kube-api-access-8px2r DeviceMajor:0 DeviceMinor:328 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c2bf2d36aec738909756a0ad609e9e57065d5b1eebfcfeba17a7ea5c518fa206/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-108 DeviceMajor:0 DeviceMinor:108 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b638dbdf-a1bf-4271-be87-4937d20d4a61/volumes/kubernetes.io~projected/kube-api-access-grqfk DeviceMajor:0 DeviceMinor:225 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8b2d52d95416e8b2657ace8a275373d23433fcc6b43c9ebe0d63d86c99966199/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c7d0befb2d6c96253f3d2eb0ce3b61b33be7c9f578898ffa084b3e50902ecfa0/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:202 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-362 DeviceMajor:0 DeviceMinor:362 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c4f3fa56-4b85-46bd-8c97-94da036ce8bb/volumes/kubernetes.io~projected/kube-api-access-zbj7f DeviceMajor:0 DeviceMinor:220 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/ca7d9994-2fc4-4328-9c50-918954121ead/volumes/kubernetes.io~projected/kube-api-access-m94sg DeviceMajor:0 DeviceMinor:58 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dc83d44b0f2991e38aa23efc9ff63fd33e647a7ac0d7090a13312cec686f34f9/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/f21610fc-0c1e-4ba3-b54e-d3db441d629f/volumes/kubernetes.io~projected/kube-api-access-zfg6f DeviceMajor:0 DeviceMinor:293 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4d145360-58fb-4376-8126-7933a7abaae2/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:203 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/8529580b-2186-4766-99d3-b55374b668ef/volumes/kubernetes.io~projected/kube-api-access-j4w5n DeviceMajor:0 DeviceMinor:217 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/074a9efa-d51e-4192-841c-3cd308d89945/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:288 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7d6e3018dea914dc3c6cb2ab7ba7835e80f68f16c53e41bafde0dd2a8c0eb182/userdata/shm DeviceMajor:0 DeviceMinor:350 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/19855996f8e578426a6a412f77cba7918b2ca057e717cdd919be47382e40f72f/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:201 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/563541bc-4a0f-4e6e-b6f0-ebd9264f537d/volumes/kubernetes.io~projected/kube-api-access-9p7nx DeviceMajor:0 DeviceMinor:221 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8/volumes/kubernetes.io~projected/kube-api-access-nhjvn DeviceMajor:0 DeviceMinor:62 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/61c81f40-4aae-4c97-9be9-30da0effd8cb/volumes/kubernetes.io~projected/kube-api-access-84hdd DeviceMajor:0 DeviceMinor:66 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:197 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/99fa95ce1a0381fe9d971595b48c44e131789da461af47cd9b6fe7d5b6f1518e/userdata/shm DeviceMajor:0 DeviceMinor:347 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fb2426a2659140ca1740b13b8a78e6e3f5ec409087f31c8d795ff2452e1f9c59/userdata/shm DeviceMajor:0 DeviceMinor:336 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/71dce3c0-a252-4070-906d-451755373263/volumes/kubernetes.io~projected/kube-api-access-b6kwf DeviceMajor:0 DeviceMinor:344 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/347e39e799415f1ae8e4989ef9a6f43d5e2630a5c377d6ec0c4e94f1cad7bcdd/userdata/shm DeviceMajor:0 DeviceMinor:346 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1afb98175d567260d50e10f134c7996aadc6aa99934f2fb26bf7638e2609e947/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/d3f48c1f-1aad-49d7-9063-f6cf60c43907/volumes/kubernetes.io~projected/kube-api-access-v2mgz DeviceMajor:0 DeviceMinor:227 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1fb59f3014e2bb210aef2057069c03ed78c84a6822166269be221c2393a8b8d3/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/cfff1c16-19ba-4492-aacb-7b75bdd64c9c/volumes/kubernetes.io~projected/kube-api-access-vl2cz DeviceMajor:0 DeviceMinor:60 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/b638dbdf-a1bf-4271-be87-4937d20d4a61/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:207 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/68217d54-7abd-4c3f-8aef-3dc2979c1014/volumes/kubernetes.io~projected/kube-api-access-h2bmc DeviceMajor:0 DeviceMinor:213 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5e15edcc9f9b1866c9b9b56e137ee21d731d367d50f40ac6bc9fbc03889771ae/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b638dbdf-a1bf-4271-be87-4937d20d4a61/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:45 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c4236bfc-5fb6-40aa-8ced-a63a86f81632/volumes/kubernetes.io~projected/kube-api-access-8x5t9 DeviceMajor:0 DeviceMinor:61 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d6aebe412b486b3379cd24a3b80fa9be586a5f39e14a501de9b49ae18378f436/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bf43b3af-6048-4d1b-9a8c-96efd387cfc4/volumes/kubernetes.io~projected/kube-api-access-7fjn6 DeviceMajor:0 DeviceMinor:214 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0/volumes/kubernetes.io~projected/kube-api-access-zlz47 DeviceMajor:0 DeviceMinor:63 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-145 DeviceMajor:0 DeviceMinor:145 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c5f64340-0f40-47b9-949c-6f25f4ef05ab/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:209 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd/volumes/kubernetes.io~projected/kube-api-access-6vdcm DeviceMajor:0 DeviceMinor:211 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5814ab61a89b534521e6eb380fe73e30473b71a8bd33e618c7c50d4970a0e85c/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6ab53da3087d52e1a1b0ef48c93ecf44e1ed66caa737f073e96aae9f70307237/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6f37acc05f224bec8051f437f9347b9f93f9ed1b29430c7f570688d6202e7e2c/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:1fb59f3014e2bb2 MacAddress:ea:c9:cd:c0:13:1b Speed:10000 Mtu:8901} {Name:30b321853ecbc0d MacAddress:c6:7a:92:92:81:ec Speed:10000 Mtu:8901} {Name:347e39e799415f1 MacAddress:32:62:1f:88:37:2b Speed:10000 Mtu:8901} {Name:5814ab61a89b534 MacAddress:fa:91:72:d7:4f:97 Speed:10000 Mtu:8901} {Name:6f37acc05f224be MacAddress:12:5b:b5:67:0f:00 Speed:10000 Mtu:8901} {Name:7d6e3018dea914d MacAddress:c2:a2:a0:86:aa:c5 Speed:10000 Mtu:8901} {Name:810344098a090bd MacAddress:42:c6:a2:90:72:26 Speed:10000 Mtu:8901} {Name:8b2d52d95416e8b MacAddress:be:7c:db:92:dd:c6 Speed:10000 Mtu:8901} {Name:99fa95ce1a0381f MacAddress:52:a4:31:19:3f:f4 Speed:10000 Mtu:8901} {Name:9da85617cf620e2 MacAddress:76:d0:c8:19:54:7a Speed:10000 Mtu:8901} {Name:b220883be33b403 MacAddress:f2:fa:e8:27:fc:bb Speed:10000 Mtu:8901} {Name:b8d25f522b81416 MacAddress:9e:e2:8d:c4:64:cc Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:dc:36:11:29 Speed:0 Mtu:9001} {Name:br-int MacAddress:36:ab:85:d8:65:a5 Speed:0 Mtu:8901} {Name:dc83d44b0f2991e MacAddress:4a:97:53:d1:3a:f7 Speed:10000 Mtu:8901} {Name:e3c3c69d9cc25f3 MacAddress:3a:27:04:e6:1d:72 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:dc:36:11:29 Speed:0 Mtu:9001} {Name:f56aa45e661ed32 MacAddress:ae:55:1d:4d:4f:89 Speed:10000 Mtu:8901} {Name:fb2426a2659140c MacAddress:02:dc:05:16:95:06 Speed:10000 Mtu:8901} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:92:c6:31:90:c0:03 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173113    5051 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173158    5051 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173388    5051 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173404    5051 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-212.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173519    5051 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173526    5051 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173535    5051 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173555    5051 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173641    5051 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173780    5051 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173881    5051 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173912    5051 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173925    5051 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173934    5051 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173942    5051 kubelet.go:405] "Adding apiserver pod source"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.173949    5051 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.174847    5051 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 19:26:39.174986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.174858    5051 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 19:26:39.175667 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.175318    5051 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 19:26:39.177601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.177587    5051 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 19:26:39.177601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.177601    5051 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 19:26:39.177703 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.177693    5051 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 03 19:26:39.177901 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.177890    5051 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 19:26:39.177934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.177919    5051 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 19:26:39.178064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178050    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 19:26:39.178115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178073    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 19:26:39.178115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178083    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 19:26:39.178115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178091    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 19:26:39.178115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178096    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 19:26:39.178115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178102    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 19:26:39.178115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178108    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 19:26:39.178115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178114    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 19:26:39.178387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178121    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 19:26:39.178387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178126    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 19:26:39.178387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178138    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 19:26:39.178387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178148    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 19:26:39.178387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178171    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 19:26:39.178387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.178177    5051 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 19:26:39.181950 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.181934    5051 server.go:1263] "Started kubelet"
Nov 03 19:26:39.182035 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.182002    5051 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 19:26:39.182190 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.182049    5051 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 19:26:39.182190 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.182149    5051 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 19:26:39.182647 ip-10-0-143-212 systemd[1]: Started Kubernetes Kubelet.
Nov 03 19:26:39.183494 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.182724    5051 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 19:26:39.183494 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.183274    5051 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 19:26:39.186985 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.186858    5051 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 19:26:39.188170 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.187209    5051 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 19:26:39.188170 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.187223    5051 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 19:26:39.188170 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.187314    5051 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 19:26:39.188170 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.187626    5051 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 19:26:39.188170 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:39.187313    5051 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-212.us-east-2.compute.internal\" not found"
Nov 03 19:26:39.188170 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.187802    5051 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 19:21:00 +0000 UTC" deadline="2028-05-31 20:28:52.023782801 +0000 UTC"
Nov 03 19:26:39.188170 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.187836    5051 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13801h2m12.835950068s"
Nov 03 19:26:39.190330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.189936    5051 factory.go:55] Registering systemd factory
Nov 03 19:26:39.190330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.189970    5051 factory.go:223] Registration of the systemd container factory successfully
Nov 03 19:26:39.190330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.190193    5051 factory.go:153] Registering CRI-O factory
Nov 03 19:26:39.190330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.190207    5051 factory.go:223] Registration of the crio container factory successfully
Nov 03 19:26:39.190330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.190274    5051 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 19:26:39.190330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.190294    5051 factory.go:103] Registering Raw factory
Nov 03 19:26:39.190330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.190308    5051 manager.go:1199] Started watching for new ooms in manager
Nov 03 19:26:39.190724 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.190705    5051 manager.go:319] Starting recovery of all containers
Nov 03 19:26:39.192179 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192125    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13fdaf11-64d1-445d-b055-fbec6b90c1bb" volumeName="kubernetes.io/configmap/13fdaf11-64d1-445d-b055-fbec6b90c1bb-telemetry-config" seLinuxMountContext=""
Nov 03 19:26:39.192289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192179    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f21610fc-0c1e-4ba3-b54e-d3db441d629f" volumeName="kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f" seLinuxMountContext=""
Nov 03 19:26:39.192289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192194    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f14faa0-564b-4e55-a3bc-984c152c5ee2" volumeName="kubernetes.io/empty-dir/5f14faa0-564b-4e55-a3bc-984c152c5ee2-archives-path" seLinuxMountContext=""
Nov 03 19:26:39.192289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192206    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="71dce3c0-a252-4070-906d-451755373263" volumeName="kubernetes.io/projected/71dce3c0-a252-4070-906d-451755373263-kube-api-access-b6kwf" seLinuxMountContext=""
Nov 03 19:26:39.192289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192218    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b638dbdf-a1bf-4271-be87-4937d20d4a61" volumeName="kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-default-certificate" seLinuxMountContext=""
Nov 03 19:26:39.192289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192231    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4f3fa56-4b85-46bd-8c97-94da036ce8bb" volumeName="kubernetes.io/projected/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-kube-api-access-zbj7f" seLinuxMountContext=""
Nov 03 19:26:39.192289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192243    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca7d9994-2fc4-4328-9c50-918954121ead" volumeName="kubernetes.io/configmap/ca7d9994-2fc4-4328-9c50-918954121ead-serviceca" seLinuxMountContext=""
Nov 03 19:26:39.192289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192256    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfff1c16-19ba-4492-aacb-7b75bdd64c9c" volumeName="kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-daemon-config" seLinuxMountContext=""
Nov 03 19:26:39.192289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192287    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ded5b685-660b-4d0f-8100-9c364ab50c7f" volumeName="kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-installation-pull-secrets" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192299    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4b3f189-03f8-4168-9e3d-b4fc806bc7cd" volumeName="kubernetes.io/secret/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-serving-cert" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192312    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14" volumeName="kubernetes.io/configmap/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-cabundle" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192324    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f14faa0-564b-4e55-a3bc-984c152c5ee2" volumeName="kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-trusted-ca-bundle" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192335    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b638dbdf-a1bf-4271-be87-4937d20d4a61" volumeName="kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-stats-auth" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192347    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4f3fa56-4b85-46bd-8c97-94da036ce8bb" volumeName="kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-snapshots" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192361    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4f3fa56-4b85-46bd-8c97-94da036ce8bb" volumeName="kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-tmp" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192373    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14" volumeName="kubernetes.io/secret/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-key" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192386    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="180203ac-0a6f-4554-8ac7-aca9b55c7b3c" volumeName="kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-script-lib" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192398    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ae31de6-e744-4e9a-a9dd-7c5203acad17" volumeName="kubernetes.io/projected/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-api-access-vwfkd" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192411    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="563541bc-4a0f-4e6e-b6f0-ebd9264f537d" volumeName="kubernetes.io/projected/563541bc-4a0f-4e6e-b6f0-ebd9264f537d-kube-api-access-9p7nx" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192422    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68217d54-7abd-4c3f-8aef-3dc2979c1014" volumeName="kubernetes.io/projected/68217d54-7abd-4c3f-8aef-3dc2979c1014-kube-api-access-h2bmc" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192435    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0" volumeName="kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-tuned" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192448    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce9d2c60-8124-4576-835d-581115d6566f" volumeName="kubernetes.io/configmap/ce9d2c60-8124-4576-835d-581115d6566f-iptables-alerter-script" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192461    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="074a9efa-d51e-4192-841c-3cd308d89945" volumeName="kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192473    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68217d54-7abd-4c3f-8aef-3dc2979c1014" volumeName="kubernetes.io/configmap/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-config" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192485    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b638dbdf-a1bf-4271-be87-4937d20d4a61" volumeName="kubernetes.io/projected/b638dbdf-a1bf-4271-be87-4937d20d4a61-kube-api-access-grqfk" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192497    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5f64340-0f40-47b9-949c-6f25f4ef05ab" volumeName="kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-config" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192509    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4a695e5-8735-4bad-8903-13c26e5bf74a" volumeName="kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-kube-api-access-kldqf" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192534    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ded5b685-660b-4d0f-8100-9c364ab50c7f" volumeName="kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-certificates" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192561    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ded5b685-660b-4d0f-8100-9c364ab50c7f" volumeName="kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-image-registry-private-configuration" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192572    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4b3f189-03f8-4168-9e3d-b4fc806bc7cd" volumeName="kubernetes.io/configmap/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-config" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192584    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10683ad8-f14a-4034-9768-bb8b168141d1" volumeName="kubernetes.io/empty-dir/10683ad8-f14a-4034-9768-bb8b168141d1-tmp-dir" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192596    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4f3fa56-4b85-46bd-8c97-94da036ce8bb" volumeName="kubernetes.io/secret/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-serving-cert" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192606    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4a695e5-8735-4bad-8903-13c26e5bf74a" volumeName="kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-installation-pull-secrets" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192617    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ded5b685-660b-4d0f-8100-9c364ab50c7f" volumeName="kubernetes.io/empty-dir/ded5b685-660b-4d0f-8100-9c364ab50c7f-ca-trust-extracted" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192629    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ded5b685-660b-4d0f-8100-9c364ab50c7f" volumeName="kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-bound-sa-token" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192640    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ae31de6-e744-4e9a-a9dd-7c5203acad17" volumeName="kubernetes.io/empty-dir/3ae31de6-e744-4e9a-a9dd-7c5203acad17-data-volume" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192652    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fd1c83c-4a87-43b1-b25e-8b9b23922aab" volumeName="kubernetes.io/secret/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-agent-certs" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192664    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f14faa0-564b-4e55-a3bc-984c152c5ee2" volumeName="kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-service-ca-bundle" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192676    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf43b3af-6048-4d1b-9a8c-96efd387cfc4" volumeName="kubernetes.io/projected/bf43b3af-6048-4d1b-9a8c-96efd387cfc4-kube-api-access-7fjn6" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192698    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4236bfc-5fb6-40aa-8ced-a63a86f81632" volumeName="kubernetes.io/projected/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kube-api-access-8x5t9" seLinuxMountContext=""
Nov 03 19:26:39.192702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192711    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5f64340-0f40-47b9-949c-6f25f4ef05ab" volumeName="kubernetes.io/projected/c5f64340-0f40-47b9-949c-6f25f4ef05ab-kube-api-access-l5tfw" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192734    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ced54a02-fff8-43e5-bebd-80a41ed74f0b" volumeName="kubernetes.io/projected/ced54a02-fff8-43e5-bebd-80a41ed74f0b-kube-api-access-rjt5p" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192747    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4a695e5-8735-4bad-8903-13c26e5bf74a" volumeName="kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-bound-sa-token" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192759    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61c81f40-4aae-4c97-9be9-30da0effd8cb" volumeName="kubernetes.io/projected/61c81f40-4aae-4c97-9be9-30da0effd8cb-kube-api-access-84hdd" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192769    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8529580b-2186-4766-99d3-b55374b668ef" volumeName="kubernetes.io/projected/8529580b-2186-4766-99d3-b55374b668ef-kube-api-access-j4w5n" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192781    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac2071b7-e0b9-4761-ae26-bc5c0ee993d8" volumeName="kubernetes.io/projected/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-kube-api-access-nhjvn" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192792    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3f48c1f-1aad-49d7-9063-f6cf60c43907" volumeName="kubernetes.io/secret/d3f48c1f-1aad-49d7-9063-f6cf60c43907-serving-cert" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192806    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4a695e5-8735-4bad-8903-13c26e5bf74a" volumeName="kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-certificates" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192819    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dd1caca1-f187-4c53-860b-5443c3b95924" volumeName="kubernetes.io/projected/dd1caca1-f187-4c53-860b-5443c3b95924-kube-api-access-59hhh" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192832    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="180203ac-0a6f-4554-8ac7-aca9b55c7b3c" volumeName="kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-env-overrides" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192844    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" volumeName="kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192856    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d145360-58fb-4376-8126-7933a7abaae2" volumeName="kubernetes.io/projected/4d145360-58fb-4376-8126-7933a7abaae2-kube-api-access-f6r2k" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192869    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f14faa0-564b-4e55-a3bc-984c152c5ee2" volumeName="kubernetes.io/projected/5f14faa0-564b-4e55-a3bc-984c152c5ee2-kube-api-access-8px2r" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192880    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="180203ac-0a6f-4554-8ac7-aca9b55c7b3c" volumeName="kubernetes.io/projected/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-kube-api-access-pjx92" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192892    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0" volumeName="kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-tmp" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192903    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0" volumeName="kubernetes.io/projected/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-kube-api-access-zlz47" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192935    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce9d2c60-8124-4576-835d-581115d6566f" volumeName="kubernetes.io/projected/ce9d2c60-8124-4576-835d-581115d6566f-kube-api-access-2dscb" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192950    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fd1c83c-4a87-43b1-b25e-8b9b23922aab" volumeName="kubernetes.io/configmap/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-konnectivity-ca" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192963    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4f3fa56-4b85-46bd-8c97-94da036ce8bb" volumeName="kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-service-ca-bundle" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192976    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4a695e5-8735-4bad-8903-13c26e5bf74a" volumeName="kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-trusted-ca" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192987    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ded5b685-660b-4d0f-8100-9c364ab50c7f" volumeName="kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-trusted-ca" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.192999    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" volumeName="kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-binary-copy" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193010    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d145360-58fb-4376-8126-7933a7abaae2" volumeName="kubernetes.io/secret/4d145360-58fb-4376-8126-7933a7abaae2-cert" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193021    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14" volumeName="kubernetes.io/projected/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-kube-api-access-nwm4z" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193035    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d9a6763-8bdf-479b-abae-a86cf51bb992" volumeName="kubernetes.io/projected/3d9a6763-8bdf-479b-abae-a86cf51bb992-kube-api-access-zpqdq" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193126    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" volumeName="kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193138    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51919e3c-99a2-484b-96ac-35948a9eec74" volumeName="kubernetes.io/configmap/51919e3c-99a2-484b-96ac-35948a9eec74-nginx-conf" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193151    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca7d9994-2fc4-4328-9c50-918954121ead" volumeName="kubernetes.io/projected/ca7d9994-2fc4-4328-9c50-918954121ead-kube-api-access-m94sg" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193162    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfff1c16-19ba-4492-aacb-7b75bdd64c9c" volumeName="kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cni-binary-copy" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193174    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ded5b685-660b-4d0f-8100-9c364ab50c7f" volumeName="kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-kube-api-access-rfdk9" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193187    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4b3f189-03f8-4168-9e3d-b4fc806bc7cd" volumeName="kubernetes.io/projected/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-kube-api-access-6vdcm" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193198    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10683ad8-f14a-4034-9768-bb8b168141d1" volumeName="kubernetes.io/projected/10683ad8-f14a-4034-9768-bb8b168141d1-kube-api-access-jvxhj" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193209    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="180203ac-0a6f-4554-8ac7-aca9b55c7b3c" volumeName="kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-config" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193221    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5f64340-0f40-47b9-949c-6f25f4ef05ab" volumeName="kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-trusted-ca" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193240    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfff1c16-19ba-4492-aacb-7b75bdd64c9c" volumeName="kubernetes.io/projected/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-kube-api-access-vl2cz" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193253    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3f48c1f-1aad-49d7-9063-f6cf60c43907" volumeName="kubernetes.io/projected/d3f48c1f-1aad-49d7-9063-f6cf60c43907-kube-api-access-v2mgz" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193267    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4a695e5-8735-4bad-8903-13c26e5bf74a" volumeName="kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-image-registry-private-configuration" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193278    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13fdaf11-64d1-445d-b055-fbec6b90c1bb" volumeName="kubernetes.io/projected/13fdaf11-64d1-445d-b055-fbec6b90c1bb-kube-api-access-b87d8" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193290    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="180203ac-0a6f-4554-8ac7-aca9b55c7b3c" volumeName="kubernetes.io/secret/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193303    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" volumeName="kubernetes.io/projected/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-kube-api-access-x86ps" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193315    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5f64340-0f40-47b9-949c-6f25f4ef05ab" volumeName="kubernetes.io/secret/c5f64340-0f40-47b9-949c-6f25f4ef05ab-serving-cert" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193329    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10683ad8-f14a-4034-9768-bb8b168141d1" volumeName="kubernetes.io/configmap/10683ad8-f14a-4034-9768-bb8b168141d1-config-volume" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193343    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61c81f40-4aae-4c97-9be9-30da0effd8cb" volumeName="kubernetes.io/empty-dir/61c81f40-4aae-4c97-9be9-30da0effd8cb-tmp-dir" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193356    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4f3fa56-4b85-46bd-8c97-94da036ce8bb" volumeName="kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-trusted-ca-bundle" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193369    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3f48c1f-1aad-49d7-9063-f6cf60c43907" volumeName="kubernetes.io/configmap/d3f48c1f-1aad-49d7-9063-f6cf60c43907-config" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193382    5051 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4a695e5-8735-4bad-8903-13c26e5bf74a" volumeName="kubernetes.io/empty-dir/d4a695e5-8735-4bad-8903-13c26e5bf74a-ca-trust-extracted" seLinuxMountContext=""
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193397    5051 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 19:26:39.194300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.193407    5051 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 19:26:39.197411 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.197388    5051 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 19:26:39.197766 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.197653    5051 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 19:26:39.203122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.200921    5051 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 19:26:39.206886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.206857    5051 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 03 19:26:39.208557 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.208534    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/1.log"
Nov 03 19:26:39.208817 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.208803    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/1.log"
Nov 03 19:26:39.219081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.219035    5051 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 19:26:39.220559 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.220539    5051 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 19:26:39.220559 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.220560    5051 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 19:26:39.220705 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.220584    5051 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 19:26:39.220705 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:39.220621    5051 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 03 19:26:39.232685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.232657    5051 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 19:26:39.233998 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.233971    5051 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="1a8551d3ab449de72ef6368ec80c20fa99b09fa061143bdeed143966edbd596e" exitCode=0
Nov 03 19:26:39.233998 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.233997    5051 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="a32e17c370667bb5f15944502e3bced1aa740e2e56d60ac72705f0bc13c1c272" exitCode=0
Nov 03 19:26:39.234156 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.234007    5051 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="bf05e7f6da51439621c6bc6386c2b53b7409b361b60697e758ae009cd7664d78" exitCode=0
Nov 03 19:26:39.234156 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.234013    5051 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="3941407d069cdfd1259b1255d65ff50340b0305414735323755585aaa523a4b6" exitCode=0
Nov 03 19:26:39.234156 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.234019    5051 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="bfcf0b75159b1caec7332a0491fe6b2091030404c3ed935ae9ff0dd0a6f7d06e" exitCode=0
Nov 03 19:26:39.234156 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.234026    5051 generic.go:356] "Generic (PLEG): container finished" podID="40a18e52-fbcd-4f04-a5f5-2ebafaea516a" containerID="4ba76dd23f7cb46ec2956118b65431abf0b569327517b8831dffb7d1d0a0800f" exitCode=0
Nov 03 19:26:39.239963 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.239944    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/1.log"
Nov 03 19:26:39.240071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.239967    5051 generic.go:356] "Generic (PLEG): container finished" podID="c5f64340-0f40-47b9-949c-6f25f4ef05ab" containerID="63aef6de7c7db6a316d45c39164997b7650203900a9a55fc42d2ff324c7e91f1" exitCode=255
Nov 03 19:26:39.280208 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.280181    5051 generic.go:356] "Generic (PLEG): container finished" podID="702b842a8006ce3b22ac9f5c693a8e83" containerID="f9444d5d8dfd60f96a7f19fa1ddbdbd1d1c405f76a83ba53558d3252fa3096bf" exitCode=0
Nov 03 19:26:39.288278 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.288262    5051 manager.go:324] Recovery completed
Nov 03 19:26:39.308797 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.308777    5051 cpu_manager.go:226] "Starting" policy="none"
Nov 03 19:26:39.308797 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.308793    5051 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 19:26:39.308923 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.308809    5051 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 19:26:39.308955 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.308934    5051 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 03 19:26:39.308983 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.308945    5051 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 03 19:26:39.308983 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.308961    5051 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 03 19:26:39.308983 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.308966    5051 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 03 19:26:39.308983 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.308973    5051 policy_none.go:50] "Start"
Nov 03 19:26:39.309136 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.308989    5051 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 19:26:39.309136 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.309001    5051 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 19:26:39.309194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.309144    5051 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 03 19:26:39.309194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.309151    5051 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 03 19:26:39.309194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.309157    5051 policy_none.go:44] "Start"
Nov 03 19:26:39.313442 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.313415    5051 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 19:26:39.316078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.316062    5051 manager.go:341] "Starting Device Plugin manager"
Nov 03 19:26:39.316523 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:39.316093    5051 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 19:26:39.316523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.316524    5051 server.go:87] "Starting device plugin registration server"
Nov 03 19:26:39.316767 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.316756    5051 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 19:26:39.316800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.316767    5051 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 19:26:39.316913 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.316889    5051 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 19:26:39.317053 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.317004    5051 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 19:26:39.317148 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.317063    5051 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 19:26:39.317148 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.317070    5051 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 19:26:39.317319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.317121    5051 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T19:26:39.31703716Z","UUID":"fb5b06bc-072c-4aa1-aad9-686f811e02ed","Handler":null,"Name":"","Endpoint":""}
Nov 03 19:26:39.318422 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.318405    5051 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 19:26:39.318519 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.318438    5051 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 19:26:39.321400 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:39.319201    5051 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 19:26:39.321531 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.321494    5051 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"]
Nov 03 19:26:39.322248 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.322232    5051 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.322454 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.322381    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal" event={"ID":"131c0f9830433abeb8e4c85b17b38230","Type":"ContainerStarted","Data":"8957fda0f81e84b1289a258b4f00fdf84464f0f06a8eeadc38332b3281166f6f"}
Nov 03 19:26:39.322454 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.322424    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal" event={"ID":"131c0f9830433abeb8e4c85b17b38230","Type":"ContainerStarted","Data":"6ab53da3087d52e1a1b0ef48c93ecf44e1ed66caa737f073e96aae9f70307237"}
Nov 03 19:26:39.322454 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.322441    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal" event={"ID":"702b842a8006ce3b22ac9f5c693a8e83","Type":"ContainerStarted","Data":"f72a7f67bca55c61dca52964e03a32a2fb025e7ba1110ef02ce3cec64e8771a4"}
Nov 03 19:26:39.322671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.322457    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal" event={"ID":"702b842a8006ce3b22ac9f5c693a8e83","Type":"ContainerDied","Data":"f9444d5d8dfd60f96a7f19fa1ddbdbd1d1c405f76a83ba53558d3252fa3096bf"}
Nov 03 19:26:39.322671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.322475    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal" event={"ID":"702b842a8006ce3b22ac9f5c693a8e83","Type":"ContainerStarted","Data":"1afb98175d567260d50e10f134c7996aadc6aa99934f2fb26bf7638e2609e947"}
Nov 03 19:26:39.322753 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.322679    5051 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.334824 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.334805    5051 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 19:26:39.334998 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:39.334979    5051 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.336190 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.336167    5051 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 19:26:39.336268 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:39.336211    5051 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.407146 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.407090    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.407146 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.407118    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.407146 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.407136    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/131c0f9830433abeb8e4c85b17b38230-config\") pod \"kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"131c0f9830433abeb8e4c85b17b38230\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.418188 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.418171    5051 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 19:26:39.419316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.419295    5051 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 19:26:39.419380 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.419328    5051 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 19:26:39.419380 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.419343    5051 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-212.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 19:26:39.419436 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.419385    5051 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.431665 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.431648    5051 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.431727 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.431700    5051 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.507566 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.507511    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.507566 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.507557    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.507736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.507601    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.507736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.507627    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/131c0f9830433abeb8e4c85b17b38230-config\") pod \"kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"131c0f9830433abeb8e4c85b17b38230\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.507736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.507676    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/131c0f9830433abeb8e4c85b17b38230-config\") pod \"kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"131c0f9830433abeb8e4c85b17b38230\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:39.507736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:39.507689    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/702b842a8006ce3b22ac9f5c693a8e83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" (UID: \"702b842a8006ce3b22ac9f5c693a8e83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:40.174744 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.174711    5051 apiserver.go:52] "Watching apiserver"
Nov 03 19:26:40.187739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.187719    5051 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 19:26:40.188449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.188425    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-2l9ws","openshift-console-operator/console-operator-7cd56b8dbc-hb5c6","openshift-dns/dns-default-nlt2r","openshift-insights/insights-operator-7c4986d449-2qjr4","openshift-ovn-kubernetes/ovnkube-node-8rkzm","openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg","openshift-image-registry/node-ca-ss5gj","openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7","openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr","kube-system/kas-connection-checker-685895c7c9-9cxfx","openshift-dns/node-resolver-rbszw","openshift-ingress-canary/ingress-canary-g9g77","openshift-insights/periodic-gathering-h9gh8-2nbhw","openshift-network-operator/iptables-alerter-5m8zz","openshift-image-registry/image-registry-7d96b8f96d-xh9sn","openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25","openshift-multus/multus-5q85z","kube-system/kas-connection-checker-685895c7c9-ctdfb","openshift-cluster-node-tuning-operator/tuned-s9jsw","openshift-insights/insights-runtime-extractor-lb4p7","openshift-multus/multus-additional-cni-plugins-jffnz","kube-system/konnectivity-agent-8hvrb","kube-system/kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5","openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r","openshift-multus/network-metrics-daemon-jfbpg","openshift-service-ca/service-ca-5fffc97bdf-fjgtn","kube-system/global-pull-secret-syncer-txdf4","openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct","openshift-ingress/router-default-687ff884df-rq9lk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal","openshift-network-console/networking-console-plugin-86c478bb88-pv5k4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9","kube-system/kas-connection-checker-685895c7c9-pxn4t","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"]
Nov 03 19:26:40.190830 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.190798    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.191282 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.191265    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.191638 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.191620    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:40.192028 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.192013    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:40.196812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.196411    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.196812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.196539    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 03 19:26:40.197239 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.197160    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wpwwc\""
Nov 03 19:26:40.197384 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.197186    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-lvj44\""
Nov 03 19:26:40.197449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.197304    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 03 19:26:40.197512 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.197489    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 03 19:26:40.197560 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.197515    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 03 19:26:40.198094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.197612    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.198094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.197974    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7rcqj\""
Nov 03 19:26:40.198094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.197997    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.198094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198025    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.198094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198060    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 03 19:26:40.198462 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198087    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.198568 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198512    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 03 19:26:40.198568 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198564    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 03 19:26:40.198692 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198606    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 03 19:26:40.198692 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198663    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.198787 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198710    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 03 19:26:40.198787 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198713    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 03 19:26:40.198787 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198737    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vjhxp\""
Nov 03 19:26:40.198921 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198789    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.198921 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198877    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.198921 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198901    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 03 19:26:40.199073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198990    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.199073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.199023    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 19:26:40.199073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.198991    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 03 19:26:40.199204 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.199077    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.199204 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.199137    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.199305 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.199253    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 03 19:26:40.199353 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.199319    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 03 19:26:40.199435 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.199409    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.199477 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.199416    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:40.199890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.199870    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 03 19:26:40.200073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.200035    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.200168 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.200151    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:40.200357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.200341    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 03 19:26:40.200451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.200435    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.200710 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.200693    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-td4dq\""
Nov 03 19:26:40.201574 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.201554    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r8qzd\""
Nov 03 19:26:40.203303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.203285    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.203621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.203604    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 03 19:26:40.203826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.203806    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 03 19:26:40.203954 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.203941    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gwvrm\""
Nov 03 19:26:40.204145 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.204125    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-4p8wb\""
Nov 03 19:26:40.204216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.204163    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 03 19:26:40.206405 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.206388    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.208502 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.208481    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-hwgf7\""
Nov 03 19:26:40.208687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.208667    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.209097 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.209080    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-dd4dw\""
Nov 03 19:26:40.209213 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.209194    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f8cx9\""
Nov 03 19:26:40.209474 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.209459    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.209569 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.209552    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 03 19:26:40.209625 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.209566    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-s8pjd\""
Nov 03 19:26:40.209782 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.209750    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.209839 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.209797    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.209959 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.209945    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.210011 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.209993    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.210266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.210249    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 19:26:40.210563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.210534    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 03 19:26:40.210710 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.210696    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xv5d7\""
Nov 03 19:26:40.210857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.210846    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 19:26:40.210933 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.210917    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 19:26:40.210981 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.210947    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.211135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.211122    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cj2kt\""
Nov 03 19:26:40.211234 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.210753    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.211419 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.211406    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ngz6s\""
Nov 03 19:26:40.211582 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.211570    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.211667 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.211599    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.211748 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.209946    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.211816 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.210252    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 03 19:26:40.211889 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.210918    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 19:26:40.211962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.211702    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:40.212098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.212081    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.212471 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.211826    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.212834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.211970    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 03 19:26:40.213101 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213080    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 19:26:40.213198 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213144    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 19:26:40.213198 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.211912    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lmsjd\""
Nov 03 19:26:40.213321 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213246    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 19:26:40.213371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213321    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 19:26:40.213371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213347    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.213455 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213421    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 03 19:26:40.213504 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213470    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 19:26:40.213557 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213537    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.213612 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213475    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.213612 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213559    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 19:26:40.213686 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213614    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 19:26:40.213732 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213680    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-st58b\""
Nov 03 19:26:40.213772 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213751    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-qhgjf\""
Nov 03 19:26:40.213918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.213903    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xp5ps\""
Nov 03 19:26:40.214028 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.214014    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.214374 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.214361    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.214429 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.214392    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 19:26:40.214758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.214740    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.214868 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.214856    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.214944 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.214870    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-xlp4b\""
Nov 03 19:26:40.214944 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.214937    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 19:26:40.215007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.214996    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.215517 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.215502    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 19:26:40.215619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.215605    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g6kk8\""
Nov 03 19:26:40.215678 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.215637    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r6b5k\""
Nov 03 19:26:40.215719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.215687    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-lsdlf\""
Nov 03 19:26:40.215767 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.215757    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 19:26:40.215808 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.215791    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-5vlcb\""
Nov 03 19:26:40.216149 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.216132    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.218716 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.218698    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.219144 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219103    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 03 19:26:40.219252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219236    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 19:26:40.219304 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219244    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 03 19:26:40.219304 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219272    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 03 19:26:40.219304 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219284    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.219397 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219312    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gh7dx\""
Nov 03 19:26:40.219506 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219495    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 03 19:26:40.219541 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219535    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 19:26:40.219592 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219577    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qmjpj\""
Nov 03 19:26:40.219624 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219598    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 19:26:40.219817 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219802    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 19:26:40.219872 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219861    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gtgvx\""
Nov 03 19:26:40.219905 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.219862    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6b2s9\""
Nov 03 19:26:40.285036 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.285010    5051 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:40.290564 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.290544    5051 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 19:26:40.296350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.296334    5051 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 19:26:40.296396 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.296379    5051 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-212.us-east-2.compute.internal"
Nov 03 19:26:40.310548 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310523    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.310664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310575    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-bound-sa-token\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.310664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310600    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-daemon-config\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.310664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310634    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-multus-certs\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.310664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310663    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-script-lib\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.310875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310688    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjx92\" (UniqueName: \"kubernetes.io/projected/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-kube-api-access-pjx92\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.310875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310722    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhjvn\" (UniqueName: \"kubernetes.io/projected/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-kube-api-access-nhjvn\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:40.310875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310751    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbj7f\" (UniqueName: \"kubernetes.io/projected/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-kube-api-access-zbj7f\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.310875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310776    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvxhj\" (UniqueName: \"kubernetes.io/projected/10683ad8-f14a-4034-9768-bb8b168141d1-kube-api-access-jvxhj\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:40.310875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310804    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2bmc\" (UniqueName: \"kubernetes.io/projected/68217d54-7abd-4c3f-8aef-3dc2979c1014-kube-api-access-h2bmc\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:40.310875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310829    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysconfig\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.310875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310850    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-tmp\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.311226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310896    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3f48c1f-1aad-49d7-9063-f6cf60c43907-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:40.311226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310926    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3f48c1f-1aad-49d7-9063-f6cf60c43907-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:40.311226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310951    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-device-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.311226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310976    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grqfk\" (UniqueName: \"kubernetes.io/projected/b638dbdf-a1bf-4271-be87-4937d20d4a61-kube-api-access-grqfk\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.311226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.310982    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.311226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311003    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-var-lib-kubelet\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.311226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311033    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-daemon-config\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.311226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311088    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-tmp\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.311226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311085    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.311226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311128    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-k8s-cni-cncf-io\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.311226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311157    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:40.311812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311412    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-serving-cert\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:40.311812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311448    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2mgz\" (UniqueName: \"kubernetes.io/projected/d3f48c1f-1aad-49d7-9063-f6cf60c43907-kube-api-access-v2mgz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:40.311812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311458    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3f48c1f-1aad-49d7-9063-f6cf60c43907-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:40.311812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311476    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-socket-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.311812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311550    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3f48c1f-1aad-49d7-9063-f6cf60c43907-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:40.311812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311603    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-bound-sa-token\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.311812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311633    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-service-ca-bundle\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.311812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311673    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-modprobe-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.311812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311698    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-bin\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.311812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311749    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-script-lib\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.312322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311836    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.312322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311881    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-etc-kubernetes\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.312322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311913    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-etc-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.312322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311944    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-serving-cert\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:40.312322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311947    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-service-ca-bundle\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.312322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.311965    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.312322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312004    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6r2k\" (UniqueName: \"kubernetes.io/projected/4d145360-58fb-4376-8126-7933a7abaae2-kube-api-access-f6r2k\") pod \"ingress-canary-g9g77\" (UID: \"4d145360-58fb-4376-8126-7933a7abaae2\") " pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:40.312322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312077    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dscb\" (UniqueName: \"kubernetes.io/projected/ce9d2c60-8124-4576-835d-581115d6566f-kube-api-access-2dscb\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:40.312322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312131    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6kwf\" (UniqueName: \"kubernetes.io/projected/71dce3c0-a252-4070-906d-451755373263-kube-api-access-b6kwf\") pod \"migrator-66df5d496b-wptct\" (UID: \"71dce3c0-a252-4070-906d-451755373263\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct"
Nov 03 19:26:40.312322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312262    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-serving-cert\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.312690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312342    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x86ps\" (UniqueName: \"kubernetes.io/projected/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-kube-api-access-x86ps\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.312690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312395    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.312690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312455    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.312690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312559    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-serving-cert\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.312690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312564    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.312690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312596    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5tfw\" (UniqueName: \"kubernetes.io/projected/c5f64340-0f40-47b9-949c-6f25f4ef05ab-kube-api-access-l5tfw\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:40.312690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312676    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-image-registry-private-configuration\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.312889 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312710    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.312889 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312783    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-trusted-ca\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.312889 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312826    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-system-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.312889 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312883    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-conf-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.313062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312921    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-cabundle\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:40.313062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312953    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-var-lib-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.313062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312980    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-image-registry-private-configuration\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.313062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.312986    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqdq\" (UniqueName: \"kubernetes.io/projected/3d9a6763-8bdf-479b-abae-a86cf51bb992-kube-api-access-zpqdq\") pod \"migrator-66df5d496b-vnqjr\" (UID: \"3d9a6763-8bdf-479b-abae-a86cf51bb992\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr"
Nov 03 19:26:40.313246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313035    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-certificates\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.313246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313164    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-cabundle\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:40.313246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313179    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-trusted-ca\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.313246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313181    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kldqf\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-kube-api-access-kldqf\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.313246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313225    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ce9d2c60-8124-4576-835d-581115d6566f-iptables-alerter-script\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:40.313428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313324    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-certificates\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.313428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313330    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-conf\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.313428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313418    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-slash\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.313549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313456    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-log-socket\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.313549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313487    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-dbus\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:40.313549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313529    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-binary-copy\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.313680 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313570    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a695e5-8735-4bad-8903-13c26e5bf74a-ca-trust-extracted\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.313680 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313587    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-systemd-units\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.313680 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313608    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-node-log\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.313680 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313616    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ce9d2c60-8124-4576-835d-581115d6566f-iptables-alerter-script\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:40.313680 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313680    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a695e5-8735-4bad-8903-13c26e5bf74a-ca-trust-extracted\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.313903 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313703    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59hhh\" (UniqueName: \"kubernetes.io/projected/dd1caca1-f187-4c53-860b-5443c3b95924-kube-api-access-59hhh\") pod \"volume-data-source-validator-58c9f5f46d-4gdd9\" (UID: \"dd1caca1-f187-4c53-860b-5443c3b95924\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9"
Nov 03 19:26:40.313903 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313733    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-key\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:40.313903 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313750    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwm4z\" (UniqueName: \"kubernetes.io/projected/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-kube-api-access-nwm4z\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:40.313903 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313802    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-config\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:40.313903 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313829    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cni-binary-copy\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.314117 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313910    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca7d9994-2fc4-4328-9c50-918954121ead-host\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:40.314117 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313975    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-signing-key\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:40.314117 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.313973    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-system-cni-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.314117 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314067    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-config\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:40.314117 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314071    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:40.314268 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314233    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51919e3c-99a2-484b-96ac-35948a9eec74-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:40.314268 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314257    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/074a9efa-d51e-4192-841c-3cd308d89945-original-pull-secret\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:40.314330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314272    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwfkd\" (UniqueName: \"kubernetes.io/projected/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-api-access-vwfkd\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.314400 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314384    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-env-overrides\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.314454 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314427    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51919e3c-99a2-484b-96ac-35948a9eec74-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:40.314531 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314507    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b87d8\" (UniqueName: \"kubernetes.io/projected/13fdaf11-64d1-445d-b055-fbec6b90c1bb-kube-api-access-b87d8\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:40.314588 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314572    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-agent-certs\") pod \"konnectivity-agent-8hvrb\" (UID: \"4fd1c83c-4a87-43b1-b25e-8b9b23922aab\") " pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:40.314631 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314604    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfg6f\" (UniqueName: \"kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f\") pod \"network-check-target-2l9ws\" (UID: \"f21610fc-0c1e-4ba3-b54e-d3db441d629f\") " pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:40.314696 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314649    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-env-overrides\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.314760 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314706    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cnibin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.314760 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314750    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-systemd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.314852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314778    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-installation-pull-secrets\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.314852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314826    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-systemd\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.314943 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314849    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-hostroot\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.314943 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314896    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.314943 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314897    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-agent-certs\") pod \"konnectivity-agent-8hvrb\" (UID: \"4fd1c83c-4a87-43b1-b25e-8b9b23922aab\") " pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:40.314943 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314922    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cnibin\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.315172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314946    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-sys-fs\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.315172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314971    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ae31de6-e744-4e9a-a9dd-7c5203acad17-crio-socket\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.315172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.314982    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-installation-pull-secrets\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.315172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315019    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-lib-modules\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.315363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315287    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-konnectivity-ca\") pod \"konnectivity-agent-8hvrb\" (UID: \"4fd1c83c-4a87-43b1-b25e-8b9b23922aab\") " pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:40.315453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315430    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ae31de6-e744-4e9a-a9dd-7c5203acad17-data-volume\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.315548 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315492    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4fd1c83c-4a87-43b1-b25e-8b9b23922aab-konnectivity-ca\") pod \"konnectivity-agent-8hvrb\" (UID: \"4fd1c83c-4a87-43b1-b25e-8b9b23922aab\") " pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:40.315548 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315513    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ae31de6-e744-4e9a-a9dd-7c5203acad17-data-volume\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.315625 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315551    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfdk9\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-kube-api-access-rfdk9\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.315625 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315594    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-kubelet\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.315708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315694    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-config\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.315752 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315737    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-service-ca-bundle\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:40.315785 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315756    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5f64340-0f40-47b9-949c-6f25f4ef05ab-serving-cert\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:40.315861 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315839    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-trusted-ca\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:40.315906 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315865    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-service-ca-bundle\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:40.315938 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315907    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-registration-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.315983 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315943    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovnkube-config\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.315983 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315948    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5f64340-0f40-47b9-949c-6f25f4ef05ab-serving-cert\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:40.316082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.315974    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-image-registry-private-configuration\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.316113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316093    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-multus\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.316174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316155    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4w5n\" (UniqueName: \"kubernetes.io/projected/8529580b-2186-4766-99d3-b55374b668ef-kube-api-access-j4w5n\") pod \"kas-connection-checker-685895c7c9-9cxfx\" (UID: \"8529580b-2186-4766-99d3-b55374b668ef\") " pod="kube-system/kas-connection-checker-685895c7c9-9cxfx"
Nov 03 19:26:40.316217 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316189    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5f14faa0-564b-4e55-a3bc-984c152c5ee2-archives-path\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:40.316217 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316211    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-image-registry-private-configuration\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.316315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316214    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m94sg\" (UniqueName: \"kubernetes.io/projected/ca7d9994-2fc4-4328-9c50-918954121ead-kube-api-access-m94sg\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:40.316315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316161    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-trusted-ca\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:40.316315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316267    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5f14faa0-564b-4e55-a3bc-984c152c5ee2-archives-path\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:40.316315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316304    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ded5b685-660b-4d0f-8100-9c364ab50c7f-ca-trust-extracted\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.316509 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316331    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61c81f40-4aae-4c97-9be9-30da0effd8cb-hosts-file\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:40.316509 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316357    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-kubernetes\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.316509 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316368    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ded5b685-660b-4d0f-8100-9c364ab50c7f-ca-trust-extracted\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.316509 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316381    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.316509 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316413    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-tuned\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.316509 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316437    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cni-binary-copy\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.316509 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316469    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-tuned\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.316509 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316481    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-stats-auth\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.316509 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316505    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-host\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.316811 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316621    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cni-binary-copy\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.316811 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316647    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-netd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.316811 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316667    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-trusted-ca-bundle\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:40.316811 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316681    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ca7d9994-2fc4-4328-9c50-918954121ead-serviceca\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:40.316811 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316697    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.316811 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316706    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-stats-auth\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.316811 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316713    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-certificates\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.317013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.316838    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-certificates\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.317086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317068    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ca7d9994-2fc4-4328-9c50-918954121ead-serviceca\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:40.317161 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317124    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlz47\" (UniqueName: \"kubernetes.io/projected/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-kube-api-access-zlz47\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.317212 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317172    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-netns\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.317263 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317210    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8px2r\" (UniqueName: \"kubernetes.io/projected/5f14faa0-564b-4e55-a3bc-984c152c5ee2-kube-api-access-8px2r\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:40.317263 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317248    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.317417 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317401    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.317492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317456    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.317555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317537    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-tmp\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.317619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317602    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-netns\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.317677 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317641    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61c81f40-4aae-4c97-9be9-30da0effd8cb-tmp-dir\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:40.317677 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317649    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-tmp\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.317775 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317691    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x5t9\" (UniqueName: \"kubernetes.io/projected/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kube-api-access-8x5t9\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.317775 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317734    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-run\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.317867 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317786    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-socket-dir-parent\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.317918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317892    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.317918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317906    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-trusted-ca-bundle\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:40.317997 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317916    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl2cz\" (UniqueName: \"kubernetes.io/projected/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-kube-api-access-vl2cz\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.317997 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.317870    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61c81f40-4aae-4c97-9be9-30da0effd8cb-tmp-dir\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:40.318085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318069    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-kubelet-config\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:40.318130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318102    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84hdd\" (UniqueName: \"kubernetes.io/projected/61c81f40-4aae-4c97-9be9-30da0effd8cb-kube-api-access-84hdd\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:40.318171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318134    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:40.318221 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318173    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-installation-pull-secrets\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.318221 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318197    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.318309 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318278    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-kubelet\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.318407 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318376    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.318514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318429    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:40.318514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318396    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-installation-pull-secrets\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.318514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318471    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovn-node-metrics-cert\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.318514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318474    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.318660 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318558    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d145360-58fb-4376-8126-7933a7abaae2-cert\") pod \"ingress-canary-g9g77\" (UID: \"4d145360-58fb-4376-8126-7933a7abaae2\") " pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:40.318660 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318591    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p7nx\" (UniqueName: \"kubernetes.io/projected/563541bc-4a0f-4e6e-b6f0-ebd9264f537d-kube-api-access-9p7nx\") pod \"kas-connection-checker-685895c7c9-pxn4t\" (UID: \"563541bc-4a0f-4e6e-b6f0-ebd9264f537d\") " pod="kube-system/kas-connection-checker-685895c7c9-pxn4t"
Nov 03 19:26:40.318660 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318619    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-trusted-ca\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.318660 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318637    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:40.318660 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318644    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:40.318887 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318677    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.318887 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318730    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/10683ad8-f14a-4034-9768-bb8b168141d1-config-volume\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:40.318887 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318783    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d145360-58fb-4376-8126-7933a7abaae2-cert\") pod \"ingress-canary-g9g77\" (UID: \"4d145360-58fb-4376-8126-7933a7abaae2\") " pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:40.318887 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318862    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-bin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.319065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318892    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-ovn\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.319065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318925    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/13fdaf11-64d1-445d-b055-fbec6b90c1bb-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:40.319065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318979    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/10683ad8-f14a-4034-9768-bb8b168141d1-config-volume\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:40.319065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.318980    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce9d2c60-8124-4576-835d-581115d6566f-host-slash\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:40.319065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319027    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-snapshots\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.319065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319027    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-ovn-node-metrics-cert\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.319065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319029    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-trusted-ca\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.319399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319085    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:40.319399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319115    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:40.319399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319115    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-snapshots\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.319399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319143    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/13fdaf11-64d1-445d-b055-fbec6b90c1bb-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:40.319399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319170    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vdcm\" (UniqueName: \"kubernetes.io/projected/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-kube-api-access-6vdcm\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:40.319399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319215    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjt5p\" (UniqueName: \"kubernetes.io/projected/ced54a02-fff8-43e5-bebd-80a41ed74f0b-kube-api-access-rjt5p\") pod \"network-check-source-6f78d59f99-sfhm7\" (UID: \"ced54a02-fff8-43e5-bebd-80a41ed74f0b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7"
Nov 03 19:26:40.319399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319300    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fjn6\" (UniqueName: \"kubernetes.io/projected/bf43b3af-6048-4d1b-9a8c-96efd387cfc4-kube-api-access-7fjn6\") pod \"kas-connection-checker-685895c7c9-ctdfb\" (UID: \"bf43b3af-6048-4d1b-9a8c-96efd387cfc4\") " pod="kube-system/kas-connection-checker-685895c7c9-ctdfb"
Nov 03 19:26:40.319399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319330    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-sys\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.319771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319426    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-config\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:40.319771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319503    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-default-certificate\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.319771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319527    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10683ad8-f14a-4034-9768-bb8b168141d1-tmp-dir\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:40.319771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319540    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5f64340-0f40-47b9-949c-6f25f4ef05ab-config\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:40.319771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319629    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10683ad8-f14a-4034-9768-bb8b168141d1-tmp-dir\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:40.319771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.319765    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-default-certificate\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.322787 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.322766    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-bound-sa-token\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.324748 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.324726    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhjvn\" (UniqueName: \"kubernetes.io/projected/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-kube-api-access-nhjvn\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:40.324901 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.324878    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2bmc\" (UniqueName: \"kubernetes.io/projected/68217d54-7abd-4c3f-8aef-3dc2979c1014-kube-api-access-h2bmc\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:40.324983 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.324923    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbj7f\" (UniqueName: \"kubernetes.io/projected/c4f3fa56-4b85-46bd-8c97-94da036ce8bb-kube-api-access-zbj7f\") pod \"insights-operator-7c4986d449-2qjr4\" (UID: \"c4f3fa56-4b85-46bd-8c97-94da036ce8bb\") " pod="openshift-insights/insights-operator-7c4986d449-2qjr4"
Nov 03 19:26:40.325061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.324983    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grqfk\" (UniqueName: \"kubernetes.io/projected/b638dbdf-a1bf-4271-be87-4937d20d4a61-kube-api-access-grqfk\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.325061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.325025    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvxhj\" (UniqueName: \"kubernetes.io/projected/10683ad8-f14a-4034-9768-bb8b168141d1-kube-api-access-jvxhj\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:40.325204 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.325187    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kldqf\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-kube-api-access-kldqf\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.325549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.325530    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dscb\" (UniqueName: \"kubernetes.io/projected/ce9d2c60-8124-4576-835d-581115d6566f-kube-api-access-2dscb\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:40.325549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.325539    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjx92\" (UniqueName: \"kubernetes.io/projected/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-kube-api-access-pjx92\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.326170 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.326148    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpqdq\" (UniqueName: \"kubernetes.io/projected/3d9a6763-8bdf-479b-abae-a86cf51bb992-kube-api-access-zpqdq\") pod \"migrator-66df5d496b-vnqjr\" (UID: \"3d9a6763-8bdf-479b-abae-a86cf51bb992\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-vnqjr"
Nov 03 19:26:40.329489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.326885    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x86ps\" (UniqueName: \"kubernetes.io/projected/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-kube-api-access-x86ps\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.329489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.327262    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-bound-sa-token\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.329489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.329118    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2mgz\" (UniqueName: \"kubernetes.io/projected/d3f48c1f-1aad-49d7-9063-f6cf60c43907-kube-api-access-v2mgz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-dpshj\" (UID: \"d3f48c1f-1aad-49d7-9063-f6cf60c43907\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj"
Nov 03 19:26:40.330849 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.329922    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5tfw\" (UniqueName: \"kubernetes.io/projected/c5f64340-0f40-47b9-949c-6f25f4ef05ab-kube-api-access-l5tfw\") pod \"console-operator-7cd56b8dbc-hb5c6\" (UID: \"c5f64340-0f40-47b9-949c-6f25f4ef05ab\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:40.330849 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.329936    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6kwf\" (UniqueName: \"kubernetes.io/projected/71dce3c0-a252-4070-906d-451755373263-kube-api-access-b6kwf\") pod \"migrator-66df5d496b-wptct\" (UID: \"71dce3c0-a252-4070-906d-451755373263\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wptct"
Nov 03 19:26:40.331679 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.331659    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6r2k\" (UniqueName: \"kubernetes.io/projected/4d145360-58fb-4376-8126-7933a7abaae2-kube-api-access-f6r2k\") pod \"ingress-canary-g9g77\" (UID: \"4d145360-58fb-4376-8126-7933a7abaae2\") " pod="openshift-ingress-canary/ingress-canary-g9g77"
Nov 03 19:26:40.332571 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.332553    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59hhh\" (UniqueName: \"kubernetes.io/projected/dd1caca1-f187-4c53-860b-5443c3b95924-kube-api-access-59hhh\") pod \"volume-data-source-validator-58c9f5f46d-4gdd9\" (UID: \"dd1caca1-f187-4c53-860b-5443c3b95924\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4gdd9"
Nov 03 19:26:40.334319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.334295    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwm4z\" (UniqueName: \"kubernetes.io/projected/0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14-kube-api-access-nwm4z\") pod \"service-ca-5fffc97bdf-fjgtn\" (UID: \"0a3b49cb-fa12-4a54-8eb7-bd7b9d14bb14\") " pod="openshift-service-ca/service-ca-5fffc97bdf-fjgtn"
Nov 03 19:26:40.334319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.334310    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwfkd\" (UniqueName: \"kubernetes.io/projected/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-api-access-vwfkd\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.334679 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.334658    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b87d8\" (UniqueName: \"kubernetes.io/projected/13fdaf11-64d1-445d-b055-fbec6b90c1bb-kube-api-access-b87d8\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:40.334816 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.334791    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfg6f\" (UniqueName: \"kubernetes.io/projected/f21610fc-0c1e-4ba3-b54e-d3db441d629f-kube-api-access-zfg6f\") pod \"network-check-target-2l9ws\" (UID: \"f21610fc-0c1e-4ba3-b54e-d3db441d629f\") " pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:40.335124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.335103    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m94sg\" (UniqueName: \"kubernetes.io/projected/ca7d9994-2fc4-4328-9c50-918954121ead-kube-api-access-m94sg\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:40.335220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.335177    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p7nx\" (UniqueName: \"kubernetes.io/projected/563541bc-4a0f-4e6e-b6f0-ebd9264f537d-kube-api-access-9p7nx\") pod \"kas-connection-checker-685895c7c9-pxn4t\" (UID: \"563541bc-4a0f-4e6e-b6f0-ebd9264f537d\") " pod="kube-system/kas-connection-checker-685895c7c9-pxn4t"
Nov 03 19:26:40.335220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.335182    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl2cz\" (UniqueName: \"kubernetes.io/projected/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-kube-api-access-vl2cz\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.335453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.335432    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vdcm\" (UniqueName: \"kubernetes.io/projected/e4b3f189-03f8-4168-9e3d-b4fc806bc7cd-kube-api-access-6vdcm\") pod \"service-ca-operator-544d749c8c-jct25\" (UID: \"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25"
Nov 03 19:26:40.335529 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.335513    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fjn6\" (UniqueName: \"kubernetes.io/projected/bf43b3af-6048-4d1b-9a8c-96efd387cfc4-kube-api-access-7fjn6\") pod \"kas-connection-checker-685895c7c9-ctdfb\" (UID: \"bf43b3af-6048-4d1b-9a8c-96efd387cfc4\") " pod="kube-system/kas-connection-checker-685895c7c9-ctdfb"
Nov 03 19:26:40.335600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.335581    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjt5p\" (UniqueName: \"kubernetes.io/projected/ced54a02-fff8-43e5-bebd-80a41ed74f0b-kube-api-access-rjt5p\") pod \"network-check-source-6f78d59f99-sfhm7\" (UID: \"ced54a02-fff8-43e5-bebd-80a41ed74f0b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-sfhm7"
Nov 03 19:26:40.336553 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.336533    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlz47\" (UniqueName: \"kubernetes.io/projected/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-kube-api-access-zlz47\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.336629 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.336560    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8px2r\" (UniqueName: \"kubernetes.io/projected/5f14faa0-564b-4e55-a3bc-984c152c5ee2-kube-api-access-8px2r\") pod \"periodic-gathering-h9gh8-2nbhw\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") " pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:26:40.336691 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.336672    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x5t9\" (UniqueName: \"kubernetes.io/projected/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kube-api-access-8x5t9\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.338633 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.338613    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfdk9\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-kube-api-access-rfdk9\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.339736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.339718    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4w5n\" (UniqueName: \"kubernetes.io/projected/8529580b-2186-4766-99d3-b55374b668ef-kube-api-access-j4w5n\") pod \"kas-connection-checker-685895c7c9-9cxfx\" (UID: \"8529580b-2186-4766-99d3-b55374b668ef\") " pod="kube-system/kas-connection-checker-685895c7c9-9cxfx"
Nov 03 19:26:40.339812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.339786    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84hdd\" (UniqueName: \"kubernetes.io/projected/61c81f40-4aae-4c97-9be9-30da0effd8cb-kube-api-access-84hdd\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:40.420423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420393    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.420423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420422    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.420593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420439    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-netns\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.420593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420453    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-run\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.420593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420484    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-socket-dir-parent\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.420593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420502    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-kubelet-config\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:40.420593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420506    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-netns\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.420593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420481    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.420593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420518    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:40.420593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420552    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-kubelet\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.420593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420561    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-socket-dir-parent\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.420593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420574    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-kubelet-config\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:40.420593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420581    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420601    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-run\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420673    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-kubelet\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420675    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420715    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.420719    5051 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.420747    5051 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420797    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-bin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.420892    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:40.920871346 +0000 UTC m=+1.828893285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.420883    5051 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.420950    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs podName:ac2071b7-e0b9-4761-ae26-bc5c0ee993d8 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:40.920922978 +0000 UTC m=+1.828944909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs") pod "network-metrics-daemon-jfbpg" (UID: "ac2071b7-e0b9-4761-ae26-bc5c0ee993d8") : secret "metrics-daemon-secret" not found
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421003    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-bin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421076    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-ovn\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421078    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ae31de6-e744-4e9a-a9dd-7c5203acad17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421113    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-ovn\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.421127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421107    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce9d2c60-8124-4576-835d-581115d6566f-host-slash\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421152    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce9d2c60-8124-4576-835d-581115d6566f-host-slash\") pod \"iptables-alerter-5m8zz\" (UID: \"ce9d2c60-8124-4576-835d-581115d6566f\") " pod="openshift-network-operator/iptables-alerter-5m8zz"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421163    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421190    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421217    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-sys\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.421244    5051 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421276    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-multus-certs\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421243    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-multus-certs\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421280    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-sys\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.421299    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls podName:10683ad8-f14a-4034-9768-bb8b168141d1 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:40.921285353 +0000 UTC m=+1.829307270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls") pod "dns-default-nlt2r" (UID: "10683ad8-f14a-4034-9768-bb8b168141d1") : secret "dns-default-metrics-tls" not found
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421337    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysconfig\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421362    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-device-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421385    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-var-lib-kubelet\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421408    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421430    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-k8s-cni-cncf-io\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421440    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-device-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421440    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysconfig\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421477    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421481    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-var-lib-kubelet\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421518    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421532    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-k8s-cni-cncf-io\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421565    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-socket-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421594    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-modprobe-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421618    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-bin\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421627    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-socket-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421646    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421688    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-bin\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421700    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-etc-kubernetes\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421725    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-etc-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421747    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-modprobe-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421749    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421789    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421796    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421837    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.421836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421842    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-etc-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421866    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421900    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421944    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.421990    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-system-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422019    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-conf-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422034    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-etc-kubernetes\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422059    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-var-lib-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422090    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-var-lib-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422115    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-conf\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422144    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-system-cni-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422143    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-slash\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422187    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-multus-conf-dir\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422187    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-log-socket\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422217    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-dbus\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422223    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-log-socket\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422250    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-systemd-units\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422278    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/074a9efa-d51e-4192-841c-3cd308d89945-dbus\") pod \"global-pull-secret-syncer-txdf4\" (UID: \"074a9efa-d51e-4192-841c-3cd308d89945\") " pod="kube-system/global-pull-secret-syncer-txdf4"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422277    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-node-log\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422309    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-node-log\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422319    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca7d9994-2fc4-4328-9c50-918954121ead-host\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422364    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca7d9994-2fc4-4328-9c50-918954121ead-host\") pod \"node-ca-ss5gj\" (UID: \"ca7d9994-2fc4-4328-9c50-918954121ead\") " pod="openshift-image-registry/node-ca-ss5gj"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422456    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-conf\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422502    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-system-cni-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422506    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422496    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b638dbdf-a1bf-4271-be87-4937d20d4a61-service-ca-bundle\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422542    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-systemd-units\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422551    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-system-cni-dir\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422556    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-slash\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422592    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cnibin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422630    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-systemd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422653    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-systemd\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422680    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-cnibin\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422676    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-hostroot\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422715    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422731    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-systemd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422743    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cnibin\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422764    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-systemd\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422771    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-run-openvswitch\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422773    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-sys-fs\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422718    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-hostroot\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422800    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ae31de6-e744-4e9a-a9dd-7c5203acad17-crio-socket\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422807    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40a18e52-fbcd-4f04-a5f5-2ebafaea516a-cnibin\") pod \"multus-additional-cni-plugins-jffnz\" (UID: \"40a18e52-fbcd-4f04-a5f5-2ebafaea516a\") " pod="openshift-multus/multus-additional-cni-plugins-jffnz"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422833    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-lib-modules\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422858    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ae31de6-e744-4e9a-a9dd-7c5203acad17-crio-socket\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422870    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-kubelet\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422835    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-sys-fs\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422894    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-registration-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422917    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-multus\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422926    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-lib-modules\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422936    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-kubelet\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422945    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61c81f40-4aae-4c97-9be9-30da0effd8cb-hosts-file\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422971    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-kubernetes\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422973    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-registration-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422993    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-var-lib-cni-multus\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.422995    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423020    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61c81f40-4aae-4c97-9be9-30da0effd8cb-hosts-file\") pod \"node-resolver-rbszw\" (UID: \"61c81f40-4aae-4c97-9be9-30da0effd8cb\") " pod="openshift-dns/node-resolver-rbszw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423029    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-host\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423071    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-netd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423089    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-kubernetes\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423098    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-etc-sysctl-d\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423098    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-netns\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423131    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cfff1c16-19ba-4492-aacb-7b75bdd64c9c-host-run-netns\") pod \"multus-5q85z\" (UID: \"cfff1c16-19ba-4492-aacb-7b75bdd64c9c\") " pod="openshift-multus/multus-5q85z"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423131    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423167    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4236bfc-5fb6-40aa-8ced-a63a86f81632-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2pww\" (UID: \"c4236bfc-5fb6-40aa-8ced-a63a86f81632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2pww"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423133    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0-host\") pod \"tuned-s9jsw\" (UID: \"97ebe2a8-5c4e-44f5-bbbe-46e49f5f55a0\") " pod="openshift-cluster-node-tuning-operator/tuned-s9jsw"
Nov 03 19:26:40.423430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423212    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/180203ac-0a6f-4554-8ac7-aca9b55c7b3c-host-cni-netd\") pod \"ovnkube-node-8rkzm\" (UID: \"180203ac-0a6f-4554-8ac7-aca9b55c7b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:40.426505 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.423791    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51919e3c-99a2-484b-96ac-35948a9eec74-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pv5k4\" (UID: \"51919e3c-99a2-484b-96ac-35948a9eec74\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:40.426505 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.424024    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68217d54-7abd-4c3f-8aef-3dc2979c1014-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6gdg5\" (UID: \"68217d54-7abd-4c3f-8aef-3dc2979c1014\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:40.426505 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.424378    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638dbdf-a1bf-4271-be87-4937d20d4a61-metrics-certs\") pod \"router-default-687ff884df-rq9lk\" (UID: \"b638dbdf-a1bf-4271-be87-4937d20d4a61\") " pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.426505 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.424584    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\") pod \"image-registry-7d96b8f96d-xh9sn\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") " pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.426505 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.425122    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13fdaf11-64d1-445d-b055-fbec6b90c1bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-x8xpg\" (UID: \"13fdaf11-64d1-445d-b055-fbec6b90c1bb\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:40.426505 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.425450    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\") pod \"image-registry-7cc7bcdcc7-r2w7r\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") " pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.491240 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.491215    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:40.492077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.492061    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"
Nov 03 19:26:40.492119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.492108    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:40.493097 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.493085    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"
Nov 03 19:26:40.495247 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.495233    5051 scope.go:122] "RemoveContainer" containerID="63aef6de7c7db6a316d45c39164997b7650203900a9a55fc42d2ff324c7e91f1"
Nov 03 19:26:40.500286 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.500268    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"
Nov 03 19:26:40.506807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.506785    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:40.806015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.805770    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-687ff884df-rq9lk"]
Nov 03 19:26:40.808452 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:40.808422    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb638dbdf_a1bf_4271_be87_4937d20d4a61.slice/crio-d328f52e59b24a52fbe1a062a4b65270c953849c8419f8fd37c25f829928f1dc WatchSource:0}: Error finding container d328f52e59b24a52fbe1a062a4b65270c953849c8419f8fd37c25f829928f1dc: Status 404 returned error can't find the container with id d328f52e59b24a52fbe1a062a4b65270c953849c8419f8fd37c25f829928f1dc
Nov 03 19:26:40.849059 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.849018    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg"]
Nov 03 19:26:40.851903 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:40.851881    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13fdaf11_64d1_445d_b055_fbec6b90c1bb.slice/crio-10dc8414124ef89fb363939de264174492bc49a92894eb11e6b243f6b8b3eb76 WatchSource:0}: Error finding container 10dc8414124ef89fb363939de264174492bc49a92894eb11e6b243f6b8b3eb76: Status 404 returned error can't find the container with id 10dc8414124ef89fb363939de264174492bc49a92894eb11e6b243f6b8b3eb76
Nov 03 19:26:40.853824 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.853807    5051 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 19:26:40.895253 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.895228    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"]
Nov 03 19:26:40.898700 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:40.898675    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded5b685_660b_4d0f_8100_9c364ab50c7f.slice/crio-347ccabadfc28eebd8c27979281651a3d8160248fb3d3d88dbc12c472148ba4e WatchSource:0}: Error finding container 347ccabadfc28eebd8c27979281651a3d8160248fb3d3d88dbc12c472148ba4e: Status 404 returned error can't find the container with id 347ccabadfc28eebd8c27979281651a3d8160248fb3d3d88dbc12c472148ba4e
Nov 03 19:26:40.928552 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.928494    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:40.928552 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.928541    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:40.928677 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.928571    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:40.928732 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.928723    5051 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 19:26:40.928793 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.928780    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls podName:10683ad8-f14a-4034-9768-bb8b168141d1 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:41.928763108 +0000 UTC m=+2.836785044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls") pod "dns-default-nlt2r" (UID: "10683ad8-f14a-4034-9768-bb8b168141d1") : secret "dns-default-metrics-tls" not found
Nov 03 19:26:40.929424 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.929401    5051 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:40.929528 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.929466    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:41.929449153 +0000 UTC m=+2.837471088 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:40.929589 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.929530    5051 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 03 19:26:40.929589 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:40.929567    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs podName:ac2071b7-e0b9-4761-ae26-bc5c0ee993d8 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:41.929555944 +0000 UTC m=+2.837577876 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs") pod "network-metrics-daemon-jfbpg" (UID: "ac2071b7-e0b9-4761-ae26-bc5c0ee993d8") : secret "metrics-daemon-secret" not found
Nov 03 19:26:40.932120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.932095    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d96b8f96d-xh9sn"]
Nov 03 19:26:40.943329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.943258    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5"]
Nov 03 19:26:40.945307 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:40.945287    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a695e5_8735_4bad_8903_13c26e5bf74a.slice/crio-6a1d44252ece886435ef7cfff8a0e467ecda429899c8e018875860d7c56d7669 WatchSource:0}: Error finding container 6a1d44252ece886435ef7cfff8a0e467ecda429899c8e018875860d7c56d7669: Status 404 returned error can't find the container with id 6a1d44252ece886435ef7cfff8a0e467ecda429899c8e018875860d7c56d7669
Nov 03 19:26:40.955760 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:40.955742    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-pv5k4"]
Nov 03 19:26:40.959856 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:40.959830    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51919e3c_99a2_484b_96ac_35948a9eec74.slice/crio-27d5d20eccd0042d54104701766b5b764d35607400a5c763a3df3e554d178217 WatchSource:0}: Error finding container 27d5d20eccd0042d54104701766b5b764d35607400a5c763a3df3e554d178217: Status 404 returned error can't find the container with id 27d5d20eccd0042d54104701766b5b764d35607400a5c763a3df3e554d178217
Nov 03 19:26:41.295257 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.295215    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r" event={"ID":"ded5b685-660b-4d0f-8100-9c364ab50c7f","Type":"ContainerStarted","Data":"571bade16307d44422df41ef363041b67ecc5fbc373ee355f9f31c00d516d660"}
Nov 03 19:26:41.295717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.295273    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r" event={"ID":"ded5b685-660b-4d0f-8100-9c364ab50c7f","Type":"ContainerStarted","Data":"347ccabadfc28eebd8c27979281651a3d8160248fb3d3d88dbc12c472148ba4e"}
Nov 03 19:26:41.296704 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.296673    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4" event={"ID":"51919e3c-99a2-484b-96ac-35948a9eec74","Type":"ContainerStarted","Data":"27d5d20eccd0042d54104701766b5b764d35607400a5c763a3df3e554d178217"}
Nov 03 19:26:41.298424 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.298392    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn" event={"ID":"d4a695e5-8735-4bad-8903-13c26e5bf74a","Type":"ContainerStarted","Data":"23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740"}
Nov 03 19:26:41.298603 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.298428    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn" event={"ID":"d4a695e5-8735-4bad-8903-13c26e5bf74a","Type":"ContainerStarted","Data":"6a1d44252ece886435ef7cfff8a0e467ecda429899c8e018875860d7c56d7669"}
Nov 03 19:26:41.300308 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.300271    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5" event={"ID":"68217d54-7abd-4c3f-8aef-3dc2979c1014","Type":"ContainerStarted","Data":"846fc918acea1dceb47033f144212379c6cca0f8137883297aabbba3c436700e"}
Nov 03 19:26:41.301941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.301923    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:26:41.302360 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.302345    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/1.log"
Nov 03 19:26:41.302433 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.302373    5051 generic.go:356] "Generic (PLEG): container finished" podID="c5f64340-0f40-47b9-949c-6f25f4ef05ab" containerID="103d150c854189affdf18f4bca52815edac8114e3de31972c327367715cf6f44" exitCode=255
Nov 03 19:26:41.302498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.302422    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6" event={"ID":"c5f64340-0f40-47b9-949c-6f25f4ef05ab","Type":"ContainerDied","Data":"103d150c854189affdf18f4bca52815edac8114e3de31972c327367715cf6f44"}
Nov 03 19:26:41.302498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.302472    5051 scope.go:122] "RemoveContainer" containerID="63aef6de7c7db6a316d45c39164997b7650203900a9a55fc42d2ff324c7e91f1"
Nov 03 19:26:41.302797 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.302779    5051 scope.go:122] "RemoveContainer" containerID="103d150c854189affdf18f4bca52815edac8114e3de31972c327367715cf6f44"
Nov 03 19:26:41.302967 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:41.302948    5051 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-hb5c6_openshift-console-operator(c5f64340-0f40-47b9-949c-6f25f4ef05ab)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6" podUID="c5f64340-0f40-47b9-949c-6f25f4ef05ab"
Nov 03 19:26:41.303753 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.303731    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg" event={"ID":"13fdaf11-64d1-445d-b055-fbec6b90c1bb","Type":"ContainerStarted","Data":"10dc8414124ef89fb363939de264174492bc49a92894eb11e6b243f6b8b3eb76"}
Nov 03 19:26:41.305515 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.305493    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-687ff884df-rq9lk" event={"ID":"b638dbdf-a1bf-4271-be87-4937d20d4a61","Type":"ContainerStarted","Data":"3db079a7c4fefbd163ee7fd8dfab24e42aaaf1fb8c25d7ea86c3193fc6b7a5c7"}
Nov 03 19:26:41.305593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.305529    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-687ff884df-rq9lk" event={"ID":"b638dbdf-a1bf-4271-be87-4937d20d4a61","Type":"ContainerStarted","Data":"d328f52e59b24a52fbe1a062a4b65270c953849c8419f8fd37c25f829928f1dc"}
Nov 03 19:26:41.491905 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.491872    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:41.494971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.494946    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:41.929354 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.929322    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:41.938715 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.938680    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:41.938860 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.938723    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:41.938860 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.938743    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:41.938860 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:41.938849    5051 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:41.939024 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:41.938925    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:43.938904832 +0000 UTC m=+4.846926765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:41.939260 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:41.938851    5051 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 19:26:41.939345 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:41.939294    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls podName:10683ad8-f14a-4034-9768-bb8b168141d1 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:43.939280573 +0000 UTC m=+4.847302503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls") pod "dns-default-nlt2r" (UID: "10683ad8-f14a-4034-9768-bb8b168141d1") : secret "dns-default-metrics-tls" not found
Nov 03 19:26:41.941633 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:41.941597    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac2071b7-e0b9-4761-ae26-bc5c0ee993d8-metrics-certs\") pod \"network-metrics-daemon-jfbpg\" (UID: \"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8\") " pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:42.001257 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:42.001225    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfbpg"
Nov 03 19:26:42.322092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:42.313971    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:26:42.322092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:42.314112    5051 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 19:26:42.322092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:42.314516    5051 scope.go:122] "RemoveContainer" containerID="103d150c854189affdf18f4bca52815edac8114e3de31972c327367715cf6f44"
Nov 03 19:26:42.322092 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:42.314712    5051 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-hb5c6_openshift-console-operator(c5f64340-0f40-47b9-949c-6f25f4ef05ab)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6" podUID="c5f64340-0f40-47b9-949c-6f25f4ef05ab"
Nov 03 19:26:42.322092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:42.315214    5051 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 19:26:42.322092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:42.315334    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:42.322092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:42.316738    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-687ff884df-rq9lk"
Nov 03 19:26:42.331432 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:42.331402    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jfbpg"]
Nov 03 19:26:42.505410 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:42.505326    5051 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:43.320304 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:43.320269    5051 generic.go:356] "Generic (PLEG): container finished" podID="e4b3f189-03f8-4168-9e3d-b4fc806bc7cd" containerID="ea926e5fc4b3c893162b3bab6524d22795d748dd1a52f7d715b7f264917f1146" exitCode=0
Nov 03 19:26:43.320457 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:43.320346    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25" event={"ID":"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd","Type":"ContainerDied","Data":"ea926e5fc4b3c893162b3bab6524d22795d748dd1a52f7d715b7f264917f1146"}
Nov 03 19:26:43.320750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:43.320730    5051 scope.go:122] "RemoveContainer" containerID="ea926e5fc4b3c893162b3bab6524d22795d748dd1a52f7d715b7f264917f1146"
Nov 03 19:26:43.321862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:43.321830    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jfbpg" event={"ID":"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8","Type":"ContainerStarted","Data":"d61208b801d9b82712f135a73225e998fff0f79f5cc68ff15d155bd20ed49618"}
Nov 03 19:26:43.323660 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:43.323630    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-pv5k4" event={"ID":"51919e3c-99a2-484b-96ac-35948a9eec74","Type":"ContainerStarted","Data":"f46acc154cdf1e48eed18befb232cf78c916ffb0b46cd6a1cf92bfb8cd39c779"}
Nov 03 19:26:43.324052 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:43.323974    5051 scope.go:122] "RemoveContainer" containerID="103d150c854189affdf18f4bca52815edac8114e3de31972c327367715cf6f44"
Nov 03 19:26:43.324193 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:43.324169    5051 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-hb5c6_openshift-console-operator(c5f64340-0f40-47b9-949c-6f25f4ef05ab)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6" podUID="c5f64340-0f40-47b9-949c-6f25f4ef05ab"
Nov 03 19:26:43.360085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:43.360059    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:43.959095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:43.958997    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:43.959095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:43.959056    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:43.959548 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:43.959530    5051 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:43.959622 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:26:43.959598    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls podName:3ae31de6-e744-4e9a-a9dd-7c5203acad17 nodeName:}" failed. No retries permitted until 2026-11-03 19:26:47.959584351 +0000 UTC m=+8.867606282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb4p7" (UID: "3ae31de6-e744-4e9a-a9dd-7c5203acad17") : secret "insights-runtime-extractor-tls" not found
Nov 03 19:26:43.961561 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:43.961538    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10683ad8-f14a-4034-9768-bb8b168141d1-metrics-tls\") pod \"dns-default-nlt2r\" (UID: \"10683ad8-f14a-4034-9768-bb8b168141d1\") " pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:44.113717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:44.113670    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:44.331089 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:44.331027    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-x8xpg" event={"ID":"13fdaf11-64d1-445d-b055-fbec6b90c1bb","Type":"ContainerStarted","Data":"c9f597759d830880b3481acf9ddf28fc7551784d57a028f3ddf944d79c76ebdc"}
Nov 03 19:26:44.333411 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:44.333385    5051 generic.go:356] "Generic (PLEG): container finished" podID="d3f48c1f-1aad-49d7-9063-f6cf60c43907" containerID="af2e9ed625e0e34dd77f82b08c80935a4c489dce53c3ee550627050d26e6d839" exitCode=0
Nov 03 19:26:44.333549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:44.333477    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj" event={"ID":"d3f48c1f-1aad-49d7-9063-f6cf60c43907","Type":"ContainerDied","Data":"af2e9ed625e0e34dd77f82b08c80935a4c489dce53c3ee550627050d26e6d839"}
Nov 03 19:26:44.333834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:44.333818    5051 scope.go:122] "RemoveContainer" containerID="af2e9ed625e0e34dd77f82b08c80935a4c489dce53c3ee550627050d26e6d839"
Nov 03 19:26:44.339763 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:44.339734    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-jct25" event={"ID":"e4b3f189-03f8-4168-9e3d-b4fc806bc7cd","Type":"ContainerStarted","Data":"97c91176007aa144a6b16034f2983c4538705408b063f7917c098c1438ebbfee"}
Nov 03 19:26:44.344225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:44.344102    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5" event={"ID":"68217d54-7abd-4c3f-8aef-3dc2979c1014","Type":"ContainerStarted","Data":"08ae0fe0e3e9325fa2aa94bd7ed0019494399c9f184af3fc9adec59d7158ac74"}
Nov 03 19:26:44.344225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:44.344154    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6gdg5" event={"ID":"68217d54-7abd-4c3f-8aef-3dc2979c1014","Type":"ContainerStarted","Data":"276da736f9dab5b1dfde4e1a07bd2d63c1053f06ea00ffa914e17816e838a115"}
Nov 03 19:26:44.581301 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:44.580849    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nlt2r"]
Nov 03 19:26:44.583595 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:44.583564    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10683ad8_f14a_4034_9768_bb8b168141d1.slice/crio-5da19176804361c2997934876022e28203aabfc18f046515fda9aacd6346b396 WatchSource:0}: Error finding container 5da19176804361c2997934876022e28203aabfc18f046515fda9aacd6346b396: Status 404 returned error can't find the container with id 5da19176804361c2997934876022e28203aabfc18f046515fda9aacd6346b396
Nov 03 19:26:45.351376 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:45.350742    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-dpshj" event={"ID":"d3f48c1f-1aad-49d7-9063-f6cf60c43907","Type":"ContainerStarted","Data":"6b749fec9a9058d3e22e26020ce1931d7134f0333e68da034086c7596399a17a"}
Nov 03 19:26:45.352632 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:45.352600    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jfbpg" event={"ID":"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8","Type":"ContainerStarted","Data":"09eabf10a02862c12b50eeea0bce1e7c35da6ab2634a36cf642666d1892d9310"}
Nov 03 19:26:45.352748 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:45.352657    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jfbpg" event={"ID":"ac2071b7-e0b9-4761-ae26-bc5c0ee993d8","Type":"ContainerStarted","Data":"47d879f95783387bb0fb0a0dac1f4c5244581e07382dd32c5a1733a76f9f240f"}
Nov 03 19:26:45.353936 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:45.353910    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nlt2r" event={"ID":"10683ad8-f14a-4034-9768-bb8b168141d1","Type":"ContainerStarted","Data":"5da19176804361c2997934876022e28203aabfc18f046515fda9aacd6346b396"}
Nov 03 19:26:46.055996 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:46.055957    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:26:46.359630 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:46.359552    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nlt2r" event={"ID":"10683ad8-f14a-4034-9768-bb8b168141d1","Type":"ContainerStarted","Data":"471ce004828e839618ea96e5e2d1e2259d5ab41a2d64b5e528276d6b5c0b78ab"}
Nov 03 19:26:46.359630 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:46.359603    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nlt2r" event={"ID":"10683ad8-f14a-4034-9768-bb8b168141d1","Type":"ContainerStarted","Data":"0505424ddf282f0c883c807f87d184559ffa227e59d3ea7e72875075db918e49"}
Nov 03 19:26:46.831223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:46.831190    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:26:47.363700 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:47.363673    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:47.661643 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:47.661561    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:47.661813 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:47.661796    5051 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 19:26:47.994840 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:47.994805    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:47.997088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:47.997064    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ae31de6-e744-4e9a-a9dd-7c5203acad17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb4p7\" (UID: \"3ae31de6-e744-4e9a-a9dd-7c5203acad17\") " pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:48.016953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:48.016927    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lb4p7"
Nov 03 19:26:48.292371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:48.292346    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lb4p7"]
Nov 03 19:26:48.294814 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:48.294788    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae31de6_e744_4e9a_a9dd_7c5203acad17.slice/crio-fc5e7b875ff25af6de7cd30bab62ccb959624bd6400de90b24cd75d80fdef827 WatchSource:0}: Error finding container fc5e7b875ff25af6de7cd30bab62ccb959624bd6400de90b24cd75d80fdef827: Status 404 returned error can't find the container with id fc5e7b875ff25af6de7cd30bab62ccb959624bd6400de90b24cd75d80fdef827
Nov 03 19:26:48.370437 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:48.370404    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lb4p7" event={"ID":"3ae31de6-e744-4e9a-a9dd-7c5203acad17","Type":"ContainerStarted","Data":"00d7b109291645b7f4c3e7b1e7c08d7b172b89630efee9fc5bbe0e5982fb7005"}
Nov 03 19:26:48.370779 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:48.370479    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lb4p7" event={"ID":"3ae31de6-e744-4e9a-a9dd-7c5203acad17","Type":"ContainerStarted","Data":"fc5e7b875ff25af6de7cd30bab62ccb959624bd6400de90b24cd75d80fdef827"}
Nov 03 19:26:48.884451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:48.884425    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rkzm"
Nov 03 19:26:49.375602 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:49.375564    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lb4p7" event={"ID":"3ae31de6-e744-4e9a-a9dd-7c5203acad17","Type":"ContainerStarted","Data":"ccd3da06c3961b6e88f315b854a13994074b22b39da89744e3bb667bb4210c4d"}
Nov 03 19:26:49.464889 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:49.464863    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:26:51.385510 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:51.385473    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lb4p7" event={"ID":"3ae31de6-e744-4e9a-a9dd-7c5203acad17","Type":"ContainerStarted","Data":"71e6ffbae1bcb95493c007d48394f5690b8758990eaa04946d599573521ae6f5"}
Nov 03 19:26:53.221883 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:53.221849    5051 scope.go:122] "RemoveContainer" containerID="103d150c854189affdf18f4bca52815edac8114e3de31972c327367715cf6f44"
Nov 03 19:26:53.394509 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:53.394483    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:26:53.394664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:53.394575    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6" event={"ID":"c5f64340-0f40-47b9-949c-6f25f4ef05ab","Type":"ContainerStarted","Data":"146dceb6ec595f82f9685e1d1ca7652e7d74befa42386a19fbba0bfabce87ee8"}
Nov 03 19:26:53.394959 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:53.394942    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:53.455852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:53.455822    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-hb5c6"
Nov 03 19:26:53.758258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:53.758227    5051 patch_prober.go:29] interesting pod/image-registry-7cc7bcdcc7-r2w7r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 03 19:26:53.758420 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:53.758277    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r" podUID="ded5b685-660b-4d0f-8100-9c364ab50c7f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 19:26:54.322457 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:54.322425    5051 patch_prober.go:29] interesting pod/image-registry-7d96b8f96d-xh9sn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 03 19:26:54.322840 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:54.322476    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn" podUID="d4a695e5-8735-4bad-8903-13c26e5bf74a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 19:26:56.060702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.060665    5051 patch_prober.go:29] interesting pod/image-registry-7cc7bcdcc7-r2w7r container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 03 19:26:56.061109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.060723    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r" podUID="ded5b685-660b-4d0f-8100-9c364ab50c7f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 19:26:56.382665 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.382632    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6"]
Nov 03 19:26:56.385073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.385034    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6"
Nov 03 19:26:56.387871 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.387847    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-r4rnd"]
Nov 03 19:26:56.389931 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.389914    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-r4rnd"
Nov 03 19:26:56.391994 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.391975    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rdqvc\""
Nov 03 19:26:56.395452 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.395431    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 03 19:26:56.400687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.400659    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 03 19:26:56.400785 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.400761    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 03 19:26:56.415364 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.415338    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-r4rnd"]
Nov 03 19:26:56.445326 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.445298    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d96b8f96d-xh9sn"]
Nov 03 19:26:56.449656 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.449631    5051 patch_prober.go:29] interesting pod/image-registry-7d96b8f96d-xh9sn container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 03 19:26:56.449804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.449670    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn" podUID="d4a695e5-8735-4bad-8903-13c26e5bf74a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 19:26:56.467980 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.467954    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d0cd4ad5-f51c-426b-912a-7a740a19879e-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-4d2p6\" (UID: \"d0cd4ad5-f51c-426b-912a-7a740a19879e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6"
Nov 03 19:26:56.468099 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.468081    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dvkl\" (UniqueName: \"kubernetes.io/projected/7f2a8a7d-2f45-4ce6-87be-08adffed4ac1-kube-api-access-9dvkl\") pod \"downloads-6dd789c8b7-r4rnd\" (UID: \"7f2a8a7d-2f45-4ce6-87be-08adffed4ac1\") " pod="openshift-console/downloads-6dd789c8b7-r4rnd"
Nov 03 19:26:56.468154 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.468120    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f2a8a7d-2f45-4ce6-87be-08adffed4ac1-tmp\") pod \"downloads-6dd789c8b7-r4rnd\" (UID: \"7f2a8a7d-2f45-4ce6-87be-08adffed4ac1\") " pod="openshift-console/downloads-6dd789c8b7-r4rnd"
Nov 03 19:26:56.474270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.474242    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6"]
Nov 03 19:26:56.492003 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.491978    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-lnv2t\""
Nov 03 19:26:56.503365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.503338    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66b954b64f-9znvn"]
Nov 03 19:26:56.505465 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.505442    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.521501 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.521480    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66b954b64f-9znvn"]
Nov 03 19:26:56.531013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.530994    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8hvrb"
Nov 03 19:26:56.568487 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.568456    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dvkl\" (UniqueName: \"kubernetes.io/projected/7f2a8a7d-2f45-4ce6-87be-08adffed4ac1-kube-api-access-9dvkl\") pod \"downloads-6dd789c8b7-r4rnd\" (UID: \"7f2a8a7d-2f45-4ce6-87be-08adffed4ac1\") " pod="openshift-console/downloads-6dd789c8b7-r4rnd"
Nov 03 19:26:56.568642 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.568497    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f2a8a7d-2f45-4ce6-87be-08adffed4ac1-tmp\") pod \"downloads-6dd789c8b7-r4rnd\" (UID: \"7f2a8a7d-2f45-4ce6-87be-08adffed4ac1\") " pod="openshift-console/downloads-6dd789c8b7-r4rnd"
Nov 03 19:26:56.568642 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.568531    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d0cd4ad5-f51c-426b-912a-7a740a19879e-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-4d2p6\" (UID: \"d0cd4ad5-f51c-426b-912a-7a740a19879e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6"
Nov 03 19:26:56.568865 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.568846    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f2a8a7d-2f45-4ce6-87be-08adffed4ac1-tmp\") pod \"downloads-6dd789c8b7-r4rnd\" (UID: \"7f2a8a7d-2f45-4ce6-87be-08adffed4ac1\") " pod="openshift-console/downloads-6dd789c8b7-r4rnd"
Nov 03 19:26:56.571133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.571105    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d0cd4ad5-f51c-426b-912a-7a740a19879e-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-4d2p6\" (UID: \"d0cd4ad5-f51c-426b-912a-7a740a19879e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6"
Nov 03 19:26:56.589288 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.589228    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dvkl\" (UniqueName: \"kubernetes.io/projected/7f2a8a7d-2f45-4ce6-87be-08adffed4ac1-kube-api-access-9dvkl\") pod \"downloads-6dd789c8b7-r4rnd\" (UID: \"7f2a8a7d-2f45-4ce6-87be-08adffed4ac1\") " pod="openshift-console/downloads-6dd789c8b7-r4rnd"
Nov 03 19:26:56.669986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.669951    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9874f516-0f3d-4220-9b89-421bd65938e6-installation-pull-secrets\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.670128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.670001    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9874f516-0f3d-4220-9b89-421bd65938e6-image-registry-private-configuration\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.670128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.670088    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9874f516-0f3d-4220-9b89-421bd65938e6-registry-certificates\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.670128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.670113    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9874f516-0f3d-4220-9b89-421bd65938e6-bound-sa-token\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.670242 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.670199    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9874f516-0f3d-4220-9b89-421bd65938e6-registry-tls\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.670277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.670266    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9874f516-0f3d-4220-9b89-421bd65938e6-ca-trust-extracted\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.670310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.670296    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4xk5\" (UniqueName: \"kubernetes.io/projected/9874f516-0f3d-4220-9b89-421bd65938e6-kube-api-access-r4xk5\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.670459 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.670440    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9874f516-0f3d-4220-9b89-421bd65938e6-trusted-ca\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.698135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.698110    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6"
Nov 03 19:26:56.703970 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.703811    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-r4rnd"
Nov 03 19:26:56.772083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.772055    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9874f516-0f3d-4220-9b89-421bd65938e6-trusted-ca\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.772280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.772112    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9874f516-0f3d-4220-9b89-421bd65938e6-installation-pull-secrets\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.772280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.772142    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9874f516-0f3d-4220-9b89-421bd65938e6-image-registry-private-configuration\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.772280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.772158    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9874f516-0f3d-4220-9b89-421bd65938e6-registry-certificates\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.772280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.772177    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9874f516-0f3d-4220-9b89-421bd65938e6-bound-sa-token\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.772280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.772199    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9874f516-0f3d-4220-9b89-421bd65938e6-registry-tls\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.772280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.772227    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9874f516-0f3d-4220-9b89-421bd65938e6-ca-trust-extracted\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.772280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.772249    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4xk5\" (UniqueName: \"kubernetes.io/projected/9874f516-0f3d-4220-9b89-421bd65938e6-kube-api-access-r4xk5\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.773135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.773106    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9874f516-0f3d-4220-9b89-421bd65938e6-trusted-ca\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.773754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.773700    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9874f516-0f3d-4220-9b89-421bd65938e6-ca-trust-extracted\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.773859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.773829    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9874f516-0f3d-4220-9b89-421bd65938e6-registry-certificates\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.775423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.775401    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9874f516-0f3d-4220-9b89-421bd65938e6-installation-pull-secrets\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.775724 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.775705    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9874f516-0f3d-4220-9b89-421bd65938e6-image-registry-private-configuration\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.775918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.775902    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9874f516-0f3d-4220-9b89-421bd65938e6-registry-tls\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.784809 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.784783    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9874f516-0f3d-4220-9b89-421bd65938e6-bound-sa-token\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.785289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.785272    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4xk5\" (UniqueName: \"kubernetes.io/projected/9874f516-0f3d-4220-9b89-421bd65938e6-kube-api-access-r4xk5\") pod \"image-registry-66b954b64f-9znvn\" (UID: \"9874f516-0f3d-4220-9b89-421bd65938e6\") " pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.817007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.816979    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:56.971752 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.971727    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6"]
Nov 03 19:26:56.974200 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:56.974146    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0cd4ad5_f51c_426b_912a_7a740a19879e.slice/crio-6ded72b79654dde5faafdd69c6f3999b496c8e13d612008f98c2640ea06113c5 WatchSource:0}: Error finding container 6ded72b79654dde5faafdd69c6f3999b496c8e13d612008f98c2640ea06113c5: Status 404 returned error can't find the container with id 6ded72b79654dde5faafdd69c6f3999b496c8e13d612008f98c2640ea06113c5
Nov 03 19:26:56.975774 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:56.975750    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-r4rnd"]
Nov 03 19:26:56.978877 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:56.978852    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f2a8a7d_2f45_4ce6_87be_08adffed4ac1.slice/crio-46f1332b82836c3de4d6913707f5ce967c55e530af2f27129da45e9f3dd9384f WatchSource:0}: Error finding container 46f1332b82836c3de4d6913707f5ce967c55e530af2f27129da45e9f3dd9384f: Status 404 returned error can't find the container with id 46f1332b82836c3de4d6913707f5ce967c55e530af2f27129da45e9f3dd9384f
Nov 03 19:26:57.100992 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:57.100960    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66b954b64f-9znvn"]
Nov 03 19:26:57.103679 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:26:57.103653    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9874f516_0f3d_4220_9b89_421bd65938e6.slice/crio-afe5270b26f24a5809965bd2b9b69f3cb37e092c316ee4c7647cf19dfa3fe499 WatchSource:0}: Error finding container afe5270b26f24a5809965bd2b9b69f3cb37e092c316ee4c7647cf19dfa3fe499: Status 404 returned error can't find the container with id afe5270b26f24a5809965bd2b9b69f3cb37e092c316ee4c7647cf19dfa3fe499
Nov 03 19:26:57.372887 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:57.372861    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nlt2r"
Nov 03 19:26:57.412896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:57.412814    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6" event={"ID":"d0cd4ad5-f51c-426b-912a-7a740a19879e","Type":"ContainerStarted","Data":"6ded72b79654dde5faafdd69c6f3999b496c8e13d612008f98c2640ea06113c5"}
Nov 03 19:26:57.414095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:57.414066    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-r4rnd" event={"ID":"7f2a8a7d-2f45-4ce6-87be-08adffed4ac1","Type":"ContainerStarted","Data":"46f1332b82836c3de4d6913707f5ce967c55e530af2f27129da45e9f3dd9384f"}
Nov 03 19:26:57.415769 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:57.415747    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b954b64f-9znvn" event={"ID":"9874f516-0f3d-4220-9b89-421bd65938e6","Type":"ContainerStarted","Data":"71324c89d10069a8c1241c4fd3afd0258fec5f3cd4c0854acf54f46949563f2f"}
Nov 03 19:26:57.415875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:57.415788    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b954b64f-9znvn" event={"ID":"9874f516-0f3d-4220-9b89-421bd65938e6","Type":"ContainerStarted","Data":"afe5270b26f24a5809965bd2b9b69f3cb37e092c316ee4c7647cf19dfa3fe499"}
Nov 03 19:26:57.415875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:57.415871    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:26:57.439260 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:57.439197    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66b954b64f-9znvn" podStartSLOduration=1.439179927 podStartE2EDuration="1.439179927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:57.438325305 +0000 UTC m=+18.346347259" watchObservedRunningTime="2026-11-03 19:26:57.439179927 +0000 UTC m=+18.347201864"
Nov 03 19:26:58.422375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:58.422293    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6" event={"ID":"d0cd4ad5-f51c-426b-912a-7a740a19879e","Type":"ContainerStarted","Data":"3a26f3cbb7aa35482076ad4017a6f04d98bbd208821989c960b9afe10d280ecb"}
Nov 03 19:26:58.422824 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:58.422528    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6"
Nov 03 19:26:58.427763 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:58.427744    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6"
Nov 03 19:26:58.444796 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:58.444742    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-4d2p6" podStartSLOduration=1.308075047 podStartE2EDuration="2.444726696s" totalImagesPullingTime="1.136651649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:26:58.444010379 +0000 UTC m=+19.352032320" watchObservedRunningTime="2026-11-03 19:26:58.444726696 +0000 UTC m=+19.352748632"
Nov 03 19:26:59.392356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.392322    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"]
Nov 03 19:26:59.395264 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.395241    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.409365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.409341    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 03 19:26:59.411971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.411092    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 03 19:26:59.411971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.411141    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wgw99\""
Nov 03 19:26:59.411971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.411145    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 03 19:26:59.413300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.413277    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"]
Nov 03 19:26:59.498853 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.498821    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6996v\" (UniqueName: \"kubernetes.io/projected/611c335f-c556-40e4-9c41-110d9b511ff1-kube-api-access-6996v\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.499227 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.498876    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/611c335f-c556-40e4-9c41-110d9b511ff1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.499227 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.498973    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/611c335f-c556-40e4-9c41-110d9b511ff1-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.499227 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.498997    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/611c335f-c556-40e4-9c41-110d9b511ff1-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.599563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.599523    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/611c335f-c556-40e4-9c41-110d9b511ff1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.599731 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.599638    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/611c335f-c556-40e4-9c41-110d9b511ff1-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.599731 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.599666    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/611c335f-c556-40e4-9c41-110d9b511ff1-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.599731 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.599710    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6996v\" (UniqueName: \"kubernetes.io/projected/611c335f-c556-40e4-9c41-110d9b511ff1-kube-api-access-6996v\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.600323 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.600299    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/611c335f-c556-40e4-9c41-110d9b511ff1-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.601980 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.601953    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/611c335f-c556-40e4-9c41-110d9b511ff1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.602088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.601952    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/611c335f-c556-40e4-9c41-110d9b511ff1-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.612315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.612294    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6996v\" (UniqueName: \"kubernetes.io/projected/611c335f-c556-40e4-9c41-110d9b511ff1-kube-api-access-6996v\") pod \"prometheus-operator-9cd89b74c-shqbr\" (UID: \"611c335f-c556-40e4-9c41-110d9b511ff1\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.718335 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.718301    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"
Nov 03 19:26:59.831717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.831682    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b8cffff76-cwxfg"]
Nov 03 19:26:59.842877 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.842848    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:26:59.849506 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.849482    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 03 19:26:59.849739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.849723    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 03 19:26:59.851173 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.851150    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 03 19:26:59.851331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.851177    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 03 19:26:59.851842 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.851824    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 03 19:26:59.853238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.853215    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b8cffff76-cwxfg"]
Nov 03 19:26:59.949661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:26:59.949630    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gzfj9\""
Nov 03 19:27:00.003545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.003136    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffdf7455-7265-4273-b344-3cf21f4a2a18-tmp\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.003545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.003196    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph9ds\" (UniqueName: \"kubernetes.io/projected/ffdf7455-7265-4273-b344-3cf21f4a2a18-kube-api-access-ph9ds\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.003545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.003227    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-oauth-config\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.003545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.003266    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-config\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.003545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.003293    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-service-ca\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.003545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.003338    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-oauth-serving-cert\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.003545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.003366    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-serving-cert\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.004693 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.004667    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-shqbr"]
Nov 03 19:27:00.008450 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:27:00.008419    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611c335f_c556_40e4_9c41_110d9b511ff1.slice/crio-27207d1906b51ee518a9bdad6f0afa0200e3243b8bfe292d2ab1f12d9f5d9984 WatchSource:0}: Error finding container 27207d1906b51ee518a9bdad6f0afa0200e3243b8bfe292d2ab1f12d9f5d9984: Status 404 returned error can't find the container with id 27207d1906b51ee518a9bdad6f0afa0200e3243b8bfe292d2ab1f12d9f5d9984
Nov 03 19:27:00.104059 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.104009    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-serving-cert\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.104252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.104110    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffdf7455-7265-4273-b344-3cf21f4a2a18-tmp\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.104252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.104158    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph9ds\" (UniqueName: \"kubernetes.io/projected/ffdf7455-7265-4273-b344-3cf21f4a2a18-kube-api-access-ph9ds\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.104252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.104188    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-oauth-config\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.104252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.104229    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-config\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.104456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.104251    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-service-ca\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.104456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.104293    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-oauth-serving-cert\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.104844 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.104818    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffdf7455-7265-4273-b344-3cf21f4a2a18-tmp\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.105180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.105136    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-config\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.105659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.105636    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-service-ca\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.106185 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.106144    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-oauth-serving-cert\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.107077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.107055    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-serving-cert\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.107077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.107070    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-oauth-config\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.117523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.117502    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph9ds\" (UniqueName: \"kubernetes.io/projected/ffdf7455-7265-4273-b344-3cf21f4a2a18-kube-api-access-ph9ds\") pod \"console-7b8cffff76-cwxfg\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") " pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.159619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.159587    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:00.433652 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.433563    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr" event={"ID":"611c335f-c556-40e4-9c41-110d9b511ff1","Type":"ContainerStarted","Data":"27207d1906b51ee518a9bdad6f0afa0200e3243b8bfe292d2ab1f12d9f5d9984"}
Nov 03 19:27:00.445291 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:00.445262    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b8cffff76-cwxfg"]
Nov 03 19:27:00.447263 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:27:00.447238    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffdf7455_7265_4273_b344_3cf21f4a2a18.slice/crio-630bd8cc90135b89fe0627fb14d5a5ec7c0e810fe5cc618326f40b721d48a766 WatchSource:0}: Error finding container 630bd8cc90135b89fe0627fb14d5a5ec7c0e810fe5cc618326f40b721d48a766: Status 404 returned error can't find the container with id 630bd8cc90135b89fe0627fb14d5a5ec7c0e810fe5cc618326f40b721d48a766
Nov 03 19:27:01.443064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:01.442958    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8cffff76-cwxfg" event={"ID":"ffdf7455-7265-4273-b344-3cf21f4a2a18","Type":"ContainerStarted","Data":"630bd8cc90135b89fe0627fb14d5a5ec7c0e810fe5cc618326f40b721d48a766"}
Nov 03 19:27:02.452247 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:02.452206    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr" event={"ID":"611c335f-c556-40e4-9c41-110d9b511ff1","Type":"ContainerStarted","Data":"f1523b9e14d76439f42a5f13cc3e304af44e9058e48bc3a3f9885209a0cd214d"}
Nov 03 19:27:02.452687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:02.452275    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr" event={"ID":"611c335f-c556-40e4-9c41-110d9b511ff1","Type":"ContainerStarted","Data":"e97108d26d4390ccab2b121631233c1d99dca854c78810dcdf971d5956856b05"}
Nov 03 19:27:02.474902 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:02.474853    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shqbr" podStartSLOduration=1.947859218 podStartE2EDuration="3.474838515s" totalImagesPullingTime="1.526979297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:27:02.47264831 +0000 UTC m=+23.380670276" watchObservedRunningTime="2026-11-03 19:27:02.474838515 +0000 UTC m=+23.382860499"
Nov 03 19:27:02.777638 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:27:02.777562    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:27:04.466867 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.464894    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8cffff76-cwxfg" event={"ID":"ffdf7455-7265-4273-b344-3cf21f4a2a18","Type":"ContainerStarted","Data":"be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d"}
Nov 03 19:27:04.492604 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.492553    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7b8cffff76-cwxfg" podStartSLOduration=2.277252543 podStartE2EDuration="5.492539147s" totalImagesPullingTime="3.215286604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:27:04.492392594 +0000 UTC m=+25.400414548" watchObservedRunningTime="2026-11-03 19:27:04.492539147 +0000 UTC m=+25.400561065"
Nov 03 19:27:04.854821 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.854738    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ghjhp"]
Nov 03 19:27:04.859465 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.859441    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:04.921821 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.921774    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 03 19:27:04.922195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.922170    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 03 19:27:04.922195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.922188    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6lz2t\""
Nov 03 19:27:04.938960 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.938932    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 03 19:27:04.954099 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.954011    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-accelerators-collector-config\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:04.954099 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.954092    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-textfile\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:04.954262 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.954179    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcgw9\" (UniqueName: \"kubernetes.io/projected/78bddb58-2426-4956-a2a2-c5bc92df3c93-kube-api-access-dcgw9\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:04.954262 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.954224    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78bddb58-2426-4956-a2a2-c5bc92df3c93-metrics-client-ca\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:04.954338 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.954280    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78bddb58-2426-4956-a2a2-c5bc92df3c93-root\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:04.954338 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.954298    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:04.954338 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.954330    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78bddb58-2426-4956-a2a2-c5bc92df3c93-sys\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:04.954452 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.954353    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-tls\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:04.954452 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:04.954375    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-wtmp\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055333 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055303    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78bddb58-2426-4956-a2a2-c5bc92df3c93-sys\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055351    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-tls\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055378    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-wtmp\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055418    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-accelerators-collector-config\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055428    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78bddb58-2426-4956-a2a2-c5bc92df3c93-sys\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055452    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-textfile\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055742 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055532    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcgw9\" (UniqueName: \"kubernetes.io/projected/78bddb58-2426-4956-a2a2-c5bc92df3c93-kube-api-access-dcgw9\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055742 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055567    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78bddb58-2426-4956-a2a2-c5bc92df3c93-metrics-client-ca\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055742 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055592    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78bddb58-2426-4956-a2a2-c5bc92df3c93-root\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055742 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055616    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-wtmp\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055742 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055617    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055983 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055797    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78bddb58-2426-4956-a2a2-c5bc92df3c93-root\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.055983 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.055802    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-textfile\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.056153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.056122    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-accelerators-collector-config\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.056566 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.056548    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78bddb58-2426-4956-a2a2-c5bc92df3c93-metrics-client-ca\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.058366 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.058342    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.058471 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.058348    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78bddb58-2426-4956-a2a2-c5bc92df3c93-node-exporter-tls\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.073553 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.073529    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcgw9\" (UniqueName: \"kubernetes.io/projected/78bddb58-2426-4956-a2a2-c5bc92df3c93-kube-api-access-dcgw9\") pod \"node-exporter-ghjhp\" (UID: \"78bddb58-2426-4956-a2a2-c5bc92df3c93\") " pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.178802 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.178722    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ghjhp"
Nov 03 19:27:05.194079 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:27:05.194033    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78bddb58_2426_4956_a2a2_c5bc92df3c93.slice/crio-e079fd45849da86ab49bf9861e3fe219dc20b7ea970f64fefd433b410005247a WatchSource:0}: Error finding container e079fd45849da86ab49bf9861e3fe219dc20b7ea970f64fefd433b410005247a: Status 404 returned error can't find the container with id e079fd45849da86ab49bf9861e3fe219dc20b7ea970f64fefd433b410005247a
Nov 03 19:27:05.471370 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:05.471329    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghjhp" event={"ID":"78bddb58-2426-4956-a2a2-c5bc92df3c93","Type":"ContainerStarted","Data":"e079fd45849da86ab49bf9861e3fe219dc20b7ea970f64fefd433b410005247a"}
Nov 03 19:27:06.062989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:06.062956    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:27:06.450605 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:06.450520    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:27:07.484444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:07.484392    5051 generic.go:356] "Generic (PLEG): container finished" podID="78bddb58-2426-4956-a2a2-c5bc92df3c93" containerID="ae1c16bdd6fa2928881b43659528d2cabaf7e8d346dfc2387237830581658194" exitCode=0
Nov 03 19:27:07.484899 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:07.484485    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghjhp" event={"ID":"78bddb58-2426-4956-a2a2-c5bc92df3c93","Type":"ContainerDied","Data":"ae1c16bdd6fa2928881b43659528d2cabaf7e8d346dfc2387237830581658194"}
Nov 03 19:27:08.492410 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:08.492368    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghjhp" event={"ID":"78bddb58-2426-4956-a2a2-c5bc92df3c93","Type":"ContainerStarted","Data":"125b178226319376cf8b96654b6e58cdfd930406890db710895d57254a97507e"}
Nov 03 19:27:08.492833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:08.492471    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ghjhp" event={"ID":"78bddb58-2426-4956-a2a2-c5bc92df3c93","Type":"ContainerStarted","Data":"85979f2a83f3631d6679ad69631ef35aefa49eef2df35f196c435523023d98b6"}
Nov 03 19:27:08.517953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:08.517903    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ghjhp" podStartSLOduration=2.829548125 podStartE2EDuration="4.517889872s" totalImagesPullingTime="1.594968676s" totalInitContainerRuntime="93.373071ms" isStatefulPod=true podCreationTimestamp="2026-11-03 19:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:27:08.515434496 +0000 UTC m=+29.423456435" watchObservedRunningTime="2026-11-03 19:27:08.517889872 +0000 UTC m=+29.425911807"
Nov 03 19:27:09.467947 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:09.467917    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2l9ws"
Nov 03 19:27:10.160869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.160609    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:10.160869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.160826    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:10.166782 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.166758    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:10.512564 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.512528    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:10.807522 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.807447    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fcc7b8c66-slr7n"]
Nov 03 19:27:10.840420 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.840378    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fcc7b8c66-slr7n"]
Nov 03 19:27:10.840600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.840510    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:10.862451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.862424    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 03 19:27:10.905399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.905364    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhkmr\" (UniqueName: \"kubernetes.io/projected/a995b7ae-83bc-44f4-b30c-1e40b473530f-kube-api-access-hhkmr\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:10.905576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.905417    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-service-ca\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:10.905576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.905441    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a995b7ae-83bc-44f4-b30c-1e40b473530f-tmp\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:10.905576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.905464    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-config\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:10.905576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.905519    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-serving-cert\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:10.905576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.905540    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-trusted-ca-bundle\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:10.905973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.905590    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-oauth-serving-cert\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:10.905973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:10.905635    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-oauth-config\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.006697 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.006654    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-service-ca\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.006697 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.006699    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a995b7ae-83bc-44f4-b30c-1e40b473530f-tmp\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.006898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.006720    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-config\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.006898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.006775    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-serving-cert\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.006898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.006798    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-trusted-ca-bundle\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.006898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.006849    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-oauth-serving-cert\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.006898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.006893    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-oauth-config\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.007066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.006935    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhkmr\" (UniqueName: \"kubernetes.io/projected/a995b7ae-83bc-44f4-b30c-1e40b473530f-kube-api-access-hhkmr\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.007498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.007450    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a995b7ae-83bc-44f4-b30c-1e40b473530f-tmp\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.008223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.007903    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-config\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.008223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.008084    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-oauth-serving-cert\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.008223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.008185    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-trusted-ca-bundle\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.008223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.008221    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-service-ca\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.010491 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.010444    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-serving-cert\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.010600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.010498    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-oauth-config\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.023929 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.023875    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhkmr\" (UniqueName: \"kubernetes.io/projected/a995b7ae-83bc-44f4-b30c-1e40b473530f-kube-api-access-hhkmr\") pod \"console-5fcc7b8c66-slr7n\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") " pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.171022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.170933    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:11.495120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.494914    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fcc7b8c66-slr7n"]
Nov 03 19:27:11.499794 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:27:11.499762    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda995b7ae_83bc_44f4_b30c_1e40b473530f.slice/crio-e3fa15df4dd796af64dcaff05ce5d213c10689cd67a5c54186ab43d96c4df2f2 WatchSource:0}: Error finding container e3fa15df4dd796af64dcaff05ce5d213c10689cd67a5c54186ab43d96c4df2f2: Status 404 returned error can't find the container with id e3fa15df4dd796af64dcaff05ce5d213c10689cd67a5c54186ab43d96c4df2f2
Nov 03 19:27:11.510622 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:11.510563    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcc7b8c66-slr7n" event={"ID":"a995b7ae-83bc-44f4-b30c-1e40b473530f","Type":"ContainerStarted","Data":"e3fa15df4dd796af64dcaff05ce5d213c10689cd67a5c54186ab43d96c4df2f2"}
Nov 03 19:27:12.517646 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:12.517604    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcc7b8c66-slr7n" event={"ID":"a995b7ae-83bc-44f4-b30c-1e40b473530f","Type":"ContainerStarted","Data":"bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c"}
Nov 03 19:27:12.539569 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:12.539515    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5fcc7b8c66-slr7n" podStartSLOduration=2.539499683 podStartE2EDuration="2.539499683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:27:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:27:12.53889683 +0000 UTC m=+33.446918770" watchObservedRunningTime="2026-11-03 19:27:12.539499683 +0000 UTC m=+33.447521623"
Nov 03 19:27:14.531075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:14.531022    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-r4rnd" event={"ID":"7f2a8a7d-2f45-4ce6-87be-08adffed4ac1","Type":"ContainerStarted","Data":"5a628d96377cedc8d47eef2e26a7705342b50a63beccae81e160b3f213c92bb2"}
Nov 03 19:27:14.531538 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:14.531250    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-r4rnd"
Nov 03 19:27:14.532845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:14.532822    5051 patch_prober.go:29] interesting pod/downloads-6dd789c8b7-r4rnd container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.29:8080/\": dial tcp 10.132.0.29:8080: connect: connection refused" start-of-body=
Nov 03 19:27:14.532897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:14.532867    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6dd789c8b7-r4rnd" podUID="7f2a8a7d-2f45-4ce6-87be-08adffed4ac1" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.29:8080/\": dial tcp 10.132.0.29:8080: connect: connection refused"
Nov 03 19:27:14.560790 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:14.560739    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-r4rnd" podStartSLOduration=1.14126434 podStartE2EDuration="18.560725389s" totalImagesPullingTime="17.419461049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:27:14.558669944 +0000 UTC m=+35.466691906" watchObservedRunningTime="2026-11-03 19:27:14.560725389 +0000 UTC m=+35.468747327"
Nov 03 19:27:15.551733 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:15.551701    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-r4rnd"
Nov 03 19:27:18.427844 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:18.427811    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66b954b64f-9znvn"
Nov 03 19:27:18.488408 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:18.488306    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"]
Nov 03 19:27:21.172072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.171960    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:21.172664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.172077    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:21.177967 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.177939    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:21.465240 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.465174    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn" podUID="d4a695e5-8735-4bad-8903-13c26e5bf74a" containerName="registry" containerID="cri-o://23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740" gracePeriod=30
Nov 03 19:27:21.572597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.572563    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:27:21.644333 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.643432    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b8cffff76-cwxfg"]
Nov 03 19:27:21.804528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.804496    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:27:21.924132 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.924096    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-kube-api-access-kldqf\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-kube-api-access-kldqf\") pod \"d4a695e5-8735-4bad-8903-13c26e5bf74a\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") "
Nov 03 19:27:21.924301 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.924159    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-trusted-ca\") pod \"d4a695e5-8735-4bad-8903-13c26e5bf74a\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") "
Nov 03 19:27:21.924301 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.924205    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4a695e5-8735-4bad-8903-13c26e5bf74a-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a695e5-8735-4bad-8903-13c26e5bf74a-ca-trust-extracted\") pod \"d4a695e5-8735-4bad-8903-13c26e5bf74a\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") "
Nov 03 19:27:21.924301 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.924275    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-certificates\") pod \"d4a695e5-8735-4bad-8903-13c26e5bf74a\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") "
Nov 03 19:27:21.924483 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.924327    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-bound-sa-token\") pod \"d4a695e5-8735-4bad-8903-13c26e5bf74a\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") "
Nov 03 19:27:21.924483 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.924359    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-installation-pull-secrets\") pod \"d4a695e5-8735-4bad-8903-13c26e5bf74a\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") "
Nov 03 19:27:21.924483 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.924439    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-image-registry-private-configuration\") pod \"d4a695e5-8735-4bad-8903-13c26e5bf74a\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") "
Nov 03 19:27:21.924483 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.924478    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\") pod \"d4a695e5-8735-4bad-8903-13c26e5bf74a\" (UID: \"d4a695e5-8735-4bad-8903-13c26e5bf74a\") "
Nov 03 19:27:21.924686 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.924661    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-certificates" pod "d4a695e5-8735-4bad-8903-13c26e5bf74a" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:27:21.925385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.924808    5051 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-certificates\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:21.926013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.925990    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-trusted-ca" pod "d4a695e5-8735-4bad-8903-13c26e5bf74a" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:27:21.927309 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.927282    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-kube-api-access-kldqf" pod "d4a695e5-8735-4bad-8903-13c26e5bf74a" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a"). InnerVolumeSpecName "kube-api-access-kldqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:27:21.928439 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.928405    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-bound-sa-token" pod "d4a695e5-8735-4bad-8903-13c26e5bf74a" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:27:21.928623 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.928578    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls" pod "d4a695e5-8735-4bad-8903-13c26e5bf74a" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:27:21.928623 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.928549    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-installation-pull-secrets" pod "d4a695e5-8735-4bad-8903-13c26e5bf74a" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:27:21.929998 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.929971    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-image-registry-private-configuration" pod "d4a695e5-8735-4bad-8903-13c26e5bf74a" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:27:21.932890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:21.932863    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a695e5-8735-4bad-8903-13c26e5bf74a-ca-trust-extracted" pod "d4a695e5-8735-4bad-8903-13c26e5bf74a" (UID: "d4a695e5-8735-4bad-8903-13c26e5bf74a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:27:22.025528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.025443    5051 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-image-registry-private-configuration\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:22.025528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.025475    5051 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-registry-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:22.025528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.025492    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kldqf\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-kube-api-access-kldqf\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:22.025528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.025506    5051 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a695e5-8735-4bad-8903-13c26e5bf74a-trusted-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:22.025528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.025520    5051 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a695e5-8735-4bad-8903-13c26e5bf74a-ca-trust-extracted\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:22.025847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.025532    5051 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a695e5-8735-4bad-8903-13c26e5bf74a-bound-sa-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:22.025847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.025549    5051 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a695e5-8735-4bad-8903-13c26e5bf74a-installation-pull-secrets\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:22.573882 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.573845    5051 generic.go:356] "Generic (PLEG): container finished" podID="d4a695e5-8735-4bad-8903-13c26e5bf74a" containerID="23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740" exitCode=0
Nov 03 19:27:22.574377 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.573899    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn"
Nov 03 19:27:22.574377 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.573934    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn" event={"ID":"d4a695e5-8735-4bad-8903-13c26e5bf74a","Type":"ContainerDied","Data":"23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740"}
Nov 03 19:27:22.574377 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.573995    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d96b8f96d-xh9sn" event={"ID":"d4a695e5-8735-4bad-8903-13c26e5bf74a","Type":"ContainerDied","Data":"6a1d44252ece886435ef7cfff8a0e467ecda429899c8e018875860d7c56d7669"}
Nov 03 19:27:22.574377 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.574024    5051 scope.go:122] "RemoveContainer" containerID="23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740"
Nov 03 19:27:22.587083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.587035    5051 scope.go:122] "RemoveContainer" containerID="23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740"
Nov 03 19:27:22.587468 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:27:22.587433    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740\": container with ID starting with 23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740 not found: ID does not exist" containerID="23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740"
Nov 03 19:27:22.587558 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.587481    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740"} err="failed to get container status \"23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740\": rpc error: code = NotFound desc = could not find container \"23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740\": container with ID starting with 23b0d9914029def2cc3c2c878fafd2cf98173a58b7804c199ba22eac3d5fd740 not found: ID does not exist"
Nov 03 19:27:22.605875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.605851    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d96b8f96d-xh9sn"]
Nov 03 19:27:22.610888 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:22.610867    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d96b8f96d-xh9sn"]
Nov 03 19:27:23.226381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:23.226341    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4a695e5-8735-4bad-8903-13c26e5bf74a" path="/var/lib/kubelet/pods/d4a695e5-8735-4bad-8903-13c26e5bf74a/volumes"
Nov 03 19:27:24.086166 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:24.086129    5051 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hvrb_4fd1c83c-4a87-43b1-b25e-8b9b23922aab/konnectivity-agent/0.log" limit=1024
Nov 03 19:27:24.222164 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:24.222126    5051 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hvrb_4fd1c83c-4a87-43b1-b25e-8b9b23922aab/konnectivity-agent/0.log" limit=1024
Nov 03 19:27:32.766263 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:27:32.766225    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:27:43.513150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.513109    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r" podUID="ded5b685-660b-4d0f-8100-9c364ab50c7f" containerName="registry" containerID="cri-o://571bade16307d44422df41ef363041b67ecc5fbc373ee355f9f31c00d516d660" gracePeriod=30
Nov 03 19:27:43.662597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.662568    5051 generic.go:356] "Generic (PLEG): container finished" podID="ded5b685-660b-4d0f-8100-9c364ab50c7f" containerID="571bade16307d44422df41ef363041b67ecc5fbc373ee355f9f31c00d516d660" exitCode=0
Nov 03 19:27:43.662739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.662649    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r" event={"ID":"ded5b685-660b-4d0f-8100-9c364ab50c7f","Type":"ContainerDied","Data":"571bade16307d44422df41ef363041b67ecc5fbc373ee355f9f31c00d516d660"}
Nov 03 19:27:43.781366 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.781307    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:27:43.914163 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914131    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-trusted-ca\") pod \"ded5b685-660b-4d0f-8100-9c364ab50c7f\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") "
Nov 03 19:27:43.914163 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914169    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-image-registry-private-configuration\") pod \"ded5b685-660b-4d0f-8100-9c364ab50c7f\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") "
Nov 03 19:27:43.914361 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914196    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-installation-pull-secrets\") pod \"ded5b685-660b-4d0f-8100-9c364ab50c7f\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") "
Nov 03 19:27:43.914361 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914223    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-kube-api-access-rfdk9\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-kube-api-access-rfdk9\") pod \"ded5b685-660b-4d0f-8100-9c364ab50c7f\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") "
Nov 03 19:27:43.914440 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914349    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-certificates\") pod \"ded5b685-660b-4d0f-8100-9c364ab50c7f\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") "
Nov 03 19:27:43.914440 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914421    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\") pod \"ded5b685-660b-4d0f-8100-9c364ab50c7f\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") "
Nov 03 19:27:43.914531 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914462    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-bound-sa-token\") pod \"ded5b685-660b-4d0f-8100-9c364ab50c7f\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") "
Nov 03 19:27:43.914595 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914530    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ded5b685-660b-4d0f-8100-9c364ab50c7f-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ded5b685-660b-4d0f-8100-9c364ab50c7f-ca-trust-extracted\") pod \"ded5b685-660b-4d0f-8100-9c364ab50c7f\" (UID: \"ded5b685-660b-4d0f-8100-9c364ab50c7f\") "
Nov 03 19:27:43.914657 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914595    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-trusted-ca" pod "ded5b685-660b-4d0f-8100-9c364ab50c7f" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:27:43.914775 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914748    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-certificates" pod "ded5b685-660b-4d0f-8100-9c364ab50c7f" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:27:43.914892 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914875    5051 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-trusted-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:43.914964 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.914899    5051 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-certificates\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:43.916686 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.916637    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls" pod "ded5b685-660b-4d0f-8100-9c364ab50c7f" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:27:43.916686 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.916646    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-kube-api-access-rfdk9" pod "ded5b685-660b-4d0f-8100-9c364ab50c7f" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f"). InnerVolumeSpecName "kube-api-access-rfdk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:27:43.916868 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.916831    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-installation-pull-secrets" pod "ded5b685-660b-4d0f-8100-9c364ab50c7f" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:27:43.917034 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.916934    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-image-registry-private-configuration" pod "ded5b685-660b-4d0f-8100-9c364ab50c7f" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:27:43.917128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.917089    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-bound-sa-token" pod "ded5b685-660b-4d0f-8100-9c364ab50c7f" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:27:43.921490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:43.921464    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded5b685-660b-4d0f-8100-9c364ab50c7f-ca-trust-extracted" pod "ded5b685-660b-4d0f-8100-9c364ab50c7f" (UID: "ded5b685-660b-4d0f-8100-9c364ab50c7f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:27:44.016262 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:44.016234    5051 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-image-registry-private-configuration\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:44.016262 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:44.016257    5051 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ded5b685-660b-4d0f-8100-9c364ab50c7f-installation-pull-secrets\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:44.016262 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:44.016268    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfdk9\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-kube-api-access-rfdk9\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:44.016444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:44.016277    5051 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-registry-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:44.016444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:44.016286    5051 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ded5b685-660b-4d0f-8100-9c364ab50c7f-bound-sa-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:44.016444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:44.016294    5051 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ded5b685-660b-4d0f-8100-9c364ab50c7f-ca-trust-extracted\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:44.667968 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:44.667936    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"
Nov 03 19:27:44.668428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:44.667971    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r" event={"ID":"ded5b685-660b-4d0f-8100-9c364ab50c7f","Type":"ContainerDied","Data":"347ccabadfc28eebd8c27979281651a3d8160248fb3d3d88dbc12c472148ba4e"}
Nov 03 19:27:44.668428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:44.668025    5051 scope.go:122] "RemoveContainer" containerID="571bade16307d44422df41ef363041b67ecc5fbc373ee355f9f31c00d516d660"
Nov 03 19:27:44.694143 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:44.694113    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"]
Nov 03 19:27:44.702395 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:44.702371    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cc7bcdcc7-r2w7r"]
Nov 03 19:27:45.228849 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:45.228814    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ded5b685-660b-4d0f-8100-9c364ab50c7f" path="/var/lib/kubelet/pods/ded5b685-660b-4d0f-8100-9c364ab50c7f/volumes"
Nov 03 19:27:46.673253 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:46.673211    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7b8cffff76-cwxfg" podUID="ffdf7455-7265-4273-b344-3cf21f4a2a18" containerName="console" containerID="cri-o://be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d" gracePeriod=15
Nov 03 19:27:46.954475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:46.954412    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:47.143855 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.143818    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-config\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-config\") pod \"ffdf7455-7265-4273-b344-3cf21f4a2a18\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") "
Nov 03 19:27:47.143855 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.143860    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-oauth-serving-cert\") pod \"ffdf7455-7265-4273-b344-3cf21f4a2a18\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") "
Nov 03 19:27:47.144130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.143888    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffdf7455-7265-4273-b344-3cf21f4a2a18-kube-api-access-ph9ds\" (UniqueName: \"kubernetes.io/projected/ffdf7455-7265-4273-b344-3cf21f4a2a18-kube-api-access-ph9ds\") pod \"ffdf7455-7265-4273-b344-3cf21f4a2a18\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") "
Nov 03 19:27:47.144130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.143941    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-serving-cert\") pod \"ffdf7455-7265-4273-b344-3cf21f4a2a18\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") "
Nov 03 19:27:47.144130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.143959    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-service-ca\") pod \"ffdf7455-7265-4273-b344-3cf21f4a2a18\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") "
Nov 03 19:27:47.144130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.144018    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffdf7455-7265-4273-b344-3cf21f4a2a18-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffdf7455-7265-4273-b344-3cf21f4a2a18-tmp\") pod \"ffdf7455-7265-4273-b344-3cf21f4a2a18\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") "
Nov 03 19:27:47.144130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.144091    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-oauth-config\") pod \"ffdf7455-7265-4273-b344-3cf21f4a2a18\" (UID: \"ffdf7455-7265-4273-b344-3cf21f4a2a18\") "
Nov 03 19:27:47.144452 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.144405    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffdf7455-7265-4273-b344-3cf21f4a2a18-tmp" pod "ffdf7455-7265-4273-b344-3cf21f4a2a18" (UID: "ffdf7455-7265-4273-b344-3cf21f4a2a18"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:27:47.144452 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.144431    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-oauth-serving-cert" pod "ffdf7455-7265-4273-b344-3cf21f4a2a18" (UID: "ffdf7455-7265-4273-b344-3cf21f4a2a18"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:27:47.144452 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.144418    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-config" pod "ffdf7455-7265-4273-b344-3cf21f4a2a18" (UID: "ffdf7455-7265-4273-b344-3cf21f4a2a18"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:27:47.144643 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.144534    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-service-ca" pod "ffdf7455-7265-4273-b344-3cf21f4a2a18" (UID: "ffdf7455-7265-4273-b344-3cf21f4a2a18"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:27:47.146358 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.146329    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-serving-cert" pod "ffdf7455-7265-4273-b344-3cf21f4a2a18" (UID: "ffdf7455-7265-4273-b344-3cf21f4a2a18"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:27:47.146471 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.146368    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-oauth-config" pod "ffdf7455-7265-4273-b344-3cf21f4a2a18" (UID: "ffdf7455-7265-4273-b344-3cf21f4a2a18"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:27:47.146471 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.146422    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffdf7455-7265-4273-b344-3cf21f4a2a18-kube-api-access-ph9ds" pod "ffdf7455-7265-4273-b344-3cf21f4a2a18" (UID: "ffdf7455-7265-4273-b344-3cf21f4a2a18"). InnerVolumeSpecName "kube-api-access-ph9ds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:27:47.244979 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.244955    5051 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-serving-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:47.244979 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.244980    5051 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:47.245143 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.244989    5051 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffdf7455-7265-4273-b344-3cf21f4a2a18-tmp\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:47.245143 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.244999    5051 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-oauth-config\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:47.245143 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.245009    5051 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-console-config\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:47.245143 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.245017    5051 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffdf7455-7265-4273-b344-3cf21f4a2a18-oauth-serving-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:47.245143 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.245026    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph9ds\" (UniqueName: \"kubernetes.io/projected/ffdf7455-7265-4273-b344-3cf21f4a2a18-kube-api-access-ph9ds\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:27:47.683191 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.683115    5051 generic.go:356] "Generic (PLEG): container finished" podID="ffdf7455-7265-4273-b344-3cf21f4a2a18" containerID="be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d" exitCode=0
Nov 03 19:27:47.683191 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.683172    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b8cffff76-cwxfg"
Nov 03 19:27:47.683695 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.683194    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8cffff76-cwxfg" event={"ID":"ffdf7455-7265-4273-b344-3cf21f4a2a18","Type":"ContainerDied","Data":"be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d"}
Nov 03 19:27:47.683695 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.683242    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b8cffff76-cwxfg" event={"ID":"ffdf7455-7265-4273-b344-3cf21f4a2a18","Type":"ContainerDied","Data":"630bd8cc90135b89fe0627fb14d5a5ec7c0e810fe5cc618326f40b721d48a766"}
Nov 03 19:27:47.683695 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.683261    5051 scope.go:122] "RemoveContainer" containerID="be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d"
Nov 03 19:27:47.691158 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.691139    5051 scope.go:122] "RemoveContainer" containerID="be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d"
Nov 03 19:27:47.691415 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:27:47.691394    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d\": container with ID starting with be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d not found: ID does not exist" containerID="be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d"
Nov 03 19:27:47.691492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.691425    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d"} err="failed to get container status \"be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d\": rpc error: code = NotFound desc = could not find container \"be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d\": container with ID starting with be79f5ba946a8838443828982c8132d30bd133637ffeb16e18423082dbb2832d not found: ID does not exist"
Nov 03 19:27:47.701698 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.701676    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b8cffff76-cwxfg"]
Nov 03 19:27:47.706598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:47.706578    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b8cffff76-cwxfg"]
Nov 03 19:27:49.226512 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:49.226471    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffdf7455-7265-4273-b344-3cf21f4a2a18" path="/var/lib/kubelet/pods/ffdf7455-7265-4273-b344-3cf21f4a2a18/volumes"
Nov 03 19:27:56.141417 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.141382    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fcdfcb4cc-hn578"]
Nov 03 19:27:56.141915 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.141841    5051 cpu_manager.go:414] "containerMap: removing container" podUID="ded5b685-660b-4d0f-8100-9c364ab50c7f" containerName="registry"
Nov 03 19:27:56.141915 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.141860    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ded5b685-660b-4d0f-8100-9c364ab50c7f" containerName="registry"
Nov 03 19:27:56.141915 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.141879    5051 cpu_manager.go:414] "containerMap: removing container" podUID="ffdf7455-7265-4273-b344-3cf21f4a2a18" containerName="console"
Nov 03 19:27:56.141915 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.141887    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffdf7455-7265-4273-b344-3cf21f4a2a18" containerName="console"
Nov 03 19:27:56.141915 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.141913    5051 cpu_manager.go:414] "containerMap: removing container" podUID="d4a695e5-8735-4bad-8903-13c26e5bf74a" containerName="registry"
Nov 03 19:27:56.142124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.141922    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4a695e5-8735-4bad-8903-13c26e5bf74a" containerName="registry"
Nov 03 19:27:56.142124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.142007    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="ded5b685-660b-4d0f-8100-9c364ab50c7f" containerName="registry"
Nov 03 19:27:56.142124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.142018    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffdf7455-7265-4273-b344-3cf21f4a2a18" containerName="console"
Nov 03 19:27:56.142124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.142030    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4a695e5-8735-4bad-8903-13c26e5bf74a" containerName="registry"
Nov 03 19:27:56.162982 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.162944    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fcdfcb4cc-hn578"]
Nov 03 19:27:56.163177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.163067    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.218444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.218415    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-oauth-config\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.218580 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.218469    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-oauth-serving-cert\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.218655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.218587    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-serving-cert\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.218655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.218618    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-console-config\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.218655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.218648    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d78e25ee-4672-4f09-b718-46a23c761ca4-tmp\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.218777 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.218703    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-service-ca\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.218777 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.218750    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-trusted-ca-bundle\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.218869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.218777    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhbmx\" (UniqueName: \"kubernetes.io/projected/d78e25ee-4672-4f09-b718-46a23c761ca4-kube-api-access-zhbmx\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.319166 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.319134    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-service-ca\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.319316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.319177    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-trusted-ca-bundle\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.319316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.319205    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhbmx\" (UniqueName: \"kubernetes.io/projected/d78e25ee-4672-4f09-b718-46a23c761ca4-kube-api-access-zhbmx\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.319316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.319247    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-oauth-config\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.319316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.319297    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-oauth-serving-cert\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.319527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.319355    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-serving-cert\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.319527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.319381    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-console-config\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.319527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.319404    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d78e25ee-4672-4f09-b718-46a23c761ca4-tmp\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.319800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.319781    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d78e25ee-4672-4f09-b718-46a23c761ca4-tmp\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.319933 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.319906    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-service-ca\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.319995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.319980    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-oauth-serving-cert\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.320136 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.320113    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-trusted-ca-bundle\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.320179 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.320132    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-console-config\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.321703 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.321682    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-oauth-config\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.321862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.321842    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-serving-cert\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.334055 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.334026    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhbmx\" (UniqueName: \"kubernetes.io/projected/d78e25ee-4672-4f09-b718-46a23c761ca4-kube-api-access-zhbmx\") pod \"console-7fcdfcb4cc-hn578\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") " pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.491798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.491774    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:27:56.798329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:56.798310    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fcdfcb4cc-hn578"]
Nov 03 19:27:57.730990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:57.730953    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fcdfcb4cc-hn578" event={"ID":"d78e25ee-4672-4f09-b718-46a23c761ca4","Type":"ContainerStarted","Data":"d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182"}
Nov 03 19:27:57.731388 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:57.731012    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fcdfcb4cc-hn578" event={"ID":"d78e25ee-4672-4f09-b718-46a23c761ca4","Type":"ContainerStarted","Data":"6052e12b6b37b30b740204a7158f897bab5376cca040e55338ecc047cdc70711"}
Nov 03 19:27:57.753777 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:27:57.753727    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7fcdfcb4cc-hn578" podStartSLOduration=1.753714676 podStartE2EDuration="1.753714676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:27:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:27:57.752796683 +0000 UTC m=+78.660818642" watchObservedRunningTime="2026-11-03 19:27:57.753714676 +0000 UTC m=+78.661736614"
Nov 03 19:28:02.767502 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:28:02.767460    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:28:03.756522 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:03.756483    5051 generic.go:356] "Generic (PLEG): container finished" podID="c4f3fa56-4b85-46bd-8c97-94da036ce8bb" containerID="b85ca642dfb8149c2ee9c07399425065ee7f6b539f46f18fff314ff5e0a60a20" exitCode=0
Nov 03 19:28:03.756706 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:03.756563    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-2qjr4" event={"ID":"c4f3fa56-4b85-46bd-8c97-94da036ce8bb","Type":"ContainerDied","Data":"b85ca642dfb8149c2ee9c07399425065ee7f6b539f46f18fff314ff5e0a60a20"}
Nov 03 19:28:03.756958 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:03.756940    5051 scope.go:122] "RemoveContainer" containerID="b85ca642dfb8149c2ee9c07399425065ee7f6b539f46f18fff314ff5e0a60a20"
Nov 03 19:28:04.300455 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.300422    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-h9gh8-2nbhw"]
Nov 03 19:28:04.300908 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.300655    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-h9gh8-2nbhw" podUID="5f14faa0-564b-4e55-a3bc-984c152c5ee2" containerName="insights-gathering" containerID="cri-o://ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27" gracePeriod=30
Nov 03 19:28:04.377951 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.377924    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-tddrc-xp7ln"]
Nov 03 19:28:04.380286 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.380265    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.391067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.391025    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-tddrc-xp7ln"]
Nov 03 19:28:04.477837 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.477802    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-service-ca-bundle\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.477837 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.477841    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-trusted-ca-bundle\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.478079 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.477969    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/474276d6-b0b6-4ac1-864a-1c57a6465bc0-archives-path\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.478079 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.477994    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dppzb\" (UniqueName: \"kubernetes.io/projected/474276d6-b0b6-4ac1-864a-1c57a6465bc0-kube-api-access-dppzb\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.551627 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.551564    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:28:04.578299 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.578272    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f14faa0-564b-4e55-a3bc-984c152c5ee2-kube-api-access-8px2r\" (UniqueName: \"kubernetes.io/projected/5f14faa0-564b-4e55-a3bc-984c152c5ee2-kube-api-access-8px2r\") pod \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") "
Nov 03 19:28:04.578447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.578334    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-trusted-ca-bundle\") pod \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") "
Nov 03 19:28:04.578447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.578363    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f14faa0-564b-4e55-a3bc-984c152c5ee2-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5f14faa0-564b-4e55-a3bc-984c152c5ee2-archives-path\") pod \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") "
Nov 03 19:28:04.578447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.578390    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-service-ca-bundle\") pod \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\" (UID: \"5f14faa0-564b-4e55-a3bc-984c152c5ee2\") "
Nov 03 19:28:04.578618 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.578537    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-service-ca-bundle\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.578618 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.578572    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-trusted-ca-bundle\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.578739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.578655    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/474276d6-b0b6-4ac1-864a-1c57a6465bc0-archives-path\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.578739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.578685    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dppzb\" (UniqueName: \"kubernetes.io/projected/474276d6-b0b6-4ac1-864a-1c57a6465bc0-kube-api-access-dppzb\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.578843 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.578771    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f14faa0-564b-4e55-a3bc-984c152c5ee2-archives-path" pod "5f14faa0-564b-4e55-a3bc-984c152c5ee2" (UID: "5f14faa0-564b-4e55-a3bc-984c152c5ee2"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:28:04.578843 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.578812    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-trusted-ca-bundle" pod "5f14faa0-564b-4e55-a3bc-984c152c5ee2" (UID: "5f14faa0-564b-4e55-a3bc-984c152c5ee2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:28:04.579346 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.579317    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-service-ca-bundle" pod "5f14faa0-564b-4e55-a3bc-984c152c5ee2" (UID: "5f14faa0-564b-4e55-a3bc-984c152c5ee2"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:28:04.579495 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.579474    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-service-ca-bundle\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.579747 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.579729    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/474276d6-b0b6-4ac1-864a-1c57a6465bc0-archives-path\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.580316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.580287    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-trusted-ca-bundle\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.581124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.581098    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f14faa0-564b-4e55-a3bc-984c152c5ee2-kube-api-access-8px2r" pod "5f14faa0-564b-4e55-a3bc-984c152c5ee2" (UID: "5f14faa0-564b-4e55-a3bc-984c152c5ee2"). InnerVolumeSpecName "kube-api-access-8px2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:28:04.591444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.591420    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dppzb\" (UniqueName: \"kubernetes.io/projected/474276d6-b0b6-4ac1-864a-1c57a6465bc0-kube-api-access-dppzb\") pod \"periodic-gathering-tddrc-xp7ln\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") " pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.679860 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.679825    5051 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-trusted-ca-bundle\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:04.679860 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.679855    5051 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5f14faa0-564b-4e55-a3bc-984c152c5ee2-archives-path\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:04.679860 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.679866    5051 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f14faa0-564b-4e55-a3bc-984c152c5ee2-service-ca-bundle\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:04.680114 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.679877    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8px2r\" (UniqueName: \"kubernetes.io/projected/5f14faa0-564b-4e55-a3bc-984c152c5ee2-kube-api-access-8px2r\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:04.699756 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.699732    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:28:04.764453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.764416    5051 generic.go:356] "Generic (PLEG): container finished" podID="5f14faa0-564b-4e55-a3bc-984c152c5ee2" containerID="ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27" exitCode=2
Nov 03 19:28:04.764565 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.764520    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-h9gh8-2nbhw" event={"ID":"5f14faa0-564b-4e55-a3bc-984c152c5ee2","Type":"ContainerDied","Data":"ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27"}
Nov 03 19:28:04.764649 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.764568    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-h9gh8-2nbhw" event={"ID":"5f14faa0-564b-4e55-a3bc-984c152c5ee2","Type":"ContainerDied","Data":"fb2426a2659140ca1740b13b8a78e6e3f5ec409087f31c8d795ff2452e1f9c59"}
Nov 03 19:28:04.764649 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.764594    5051 scope.go:122] "RemoveContainer" containerID="ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27"
Nov 03 19:28:04.764738 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.764724    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-h9gh8-2nbhw"
Nov 03 19:28:04.768829 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.768381    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-2qjr4" event={"ID":"c4f3fa56-4b85-46bd-8c97-94da036ce8bb","Type":"ContainerStarted","Data":"98f8efaf6cd4fab73f52e5dc938dc52eef26d910ca7932a23e905754ef23e709"}
Nov 03 19:28:04.774834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.774817    5051 scope.go:122] "RemoveContainer" containerID="ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27"
Nov 03 19:28:04.775271 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:28:04.775114    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27\": container with ID starting with ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27 not found: ID does not exist" containerID="ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27"
Nov 03 19:28:04.775271 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.775151    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27"} err="failed to get container status \"ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27\": rpc error: code = NotFound desc = could not find container \"ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27\": container with ID starting with ee968095f5cb8fd07d82c3c0e179d0f4d77785390a2d835bce427c5d1b2cbc27 not found: ID does not exist"
Nov 03 19:28:04.809585 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.809521    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-h9gh8-2nbhw"]
Nov 03 19:28:04.814253 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.814227    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-h9gh8-2nbhw"]
Nov 03 19:28:04.971755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:04.971733    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-tddrc-xp7ln"]
Nov 03 19:28:04.973724 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:28:04.973696    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod474276d6_b0b6_4ac1_864a_1c57a6465bc0.slice/crio-5401cca2317051740319f60d278ff4f337d10d29691a1c8d41705023f641954d WatchSource:0}: Error finding container 5401cca2317051740319f60d278ff4f337d10d29691a1c8d41705023f641954d: Status 404 returned error can't find the container with id 5401cca2317051740319f60d278ff4f337d10d29691a1c8d41705023f641954d
Nov 03 19:28:05.226996 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:05.226963    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f14faa0-564b-4e55-a3bc-984c152c5ee2" path="/var/lib/kubelet/pods/5f14faa0-564b-4e55-a3bc-984c152c5ee2/volumes"
Nov 03 19:28:05.774493 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:05.774457    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-tddrc-xp7ln" event={"ID":"474276d6-b0b6-4ac1-864a-1c57a6465bc0","Type":"ContainerStarted","Data":"f2920ee25c369f87061106c3de216598da717bb2ecc3b4d6972a790aea46a9d7"}
Nov 03 19:28:05.774961 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:05.774515    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-tddrc-xp7ln" event={"ID":"474276d6-b0b6-4ac1-864a-1c57a6465bc0","Type":"ContainerStarted","Data":"5401cca2317051740319f60d278ff4f337d10d29691a1c8d41705023f641954d"}
Nov 03 19:28:05.793446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:05.793404    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-tddrc-xp7ln" podStartSLOduration=1.7933922089999998 podStartE2EDuration="1.793392209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:28:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:28:05.792977532 +0000 UTC m=+86.700999470" watchObservedRunningTime="2026-11-03 19:28:05.793392209 +0000 UTC m=+86.701414147"
Nov 03 19:28:06.491987 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:06.491953    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:28:06.492194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:06.492022    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:28:06.496640 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:06.496619    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:28:06.782874 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:06.782797    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:28:06.838277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:06.838235    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fcc7b8c66-slr7n"]
Nov 03 19:28:31.872221 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:31.872160    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5fcc7b8c66-slr7n" podUID="a995b7ae-83bc-44f4-b30c-1e40b473530f" containerName="console" containerID="cri-o://bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c" gracePeriod=15
Nov 03 19:28:32.115258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.115235    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:28:32.200600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.200528    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-oauth-config\") pod \"a995b7ae-83bc-44f4-b30c-1e40b473530f\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") "
Nov 03 19:28:32.200600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.200565    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-trusted-ca-bundle\") pod \"a995b7ae-83bc-44f4-b30c-1e40b473530f\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") "
Nov 03 19:28:32.200778 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.200613    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-config\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-config\") pod \"a995b7ae-83bc-44f4-b30c-1e40b473530f\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") "
Nov 03 19:28:32.200778 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.200650    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a995b7ae-83bc-44f4-b30c-1e40b473530f-kube-api-access-hhkmr\" (UniqueName: \"kubernetes.io/projected/a995b7ae-83bc-44f4-b30c-1e40b473530f-kube-api-access-hhkmr\") pod \"a995b7ae-83bc-44f4-b30c-1e40b473530f\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") "
Nov 03 19:28:32.200778 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.200666    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-serving-cert\") pod \"a995b7ae-83bc-44f4-b30c-1e40b473530f\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") "
Nov 03 19:28:32.200778 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.200686    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-service-ca\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-service-ca\") pod \"a995b7ae-83bc-44f4-b30c-1e40b473530f\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") "
Nov 03 19:28:32.200778 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.200710    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-oauth-serving-cert\") pod \"a995b7ae-83bc-44f4-b30c-1e40b473530f\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") "
Nov 03 19:28:32.201128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.200778    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a995b7ae-83bc-44f4-b30c-1e40b473530f-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a995b7ae-83bc-44f4-b30c-1e40b473530f-tmp\") pod \"a995b7ae-83bc-44f4-b30c-1e40b473530f\" (UID: \"a995b7ae-83bc-44f4-b30c-1e40b473530f\") "
Nov 03 19:28:32.201128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.201099    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-config" pod "a995b7ae-83bc-44f4-b30c-1e40b473530f" (UID: "a995b7ae-83bc-44f4-b30c-1e40b473530f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:28:32.201259 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.201121    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-trusted-ca-bundle" pod "a995b7ae-83bc-44f4-b30c-1e40b473530f" (UID: "a995b7ae-83bc-44f4-b30c-1e40b473530f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:28:32.201314 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.201261    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a995b7ae-83bc-44f4-b30c-1e40b473530f-tmp" pod "a995b7ae-83bc-44f4-b30c-1e40b473530f" (UID: "a995b7ae-83bc-44f4-b30c-1e40b473530f"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:28:32.201381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.201354    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-oauth-serving-cert" pod "a995b7ae-83bc-44f4-b30c-1e40b473530f" (UID: "a995b7ae-83bc-44f4-b30c-1e40b473530f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:28:32.201442 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.201399    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-service-ca" pod "a995b7ae-83bc-44f4-b30c-1e40b473530f" (UID: "a995b7ae-83bc-44f4-b30c-1e40b473530f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:28:32.202733 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.202710    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-oauth-config" pod "a995b7ae-83bc-44f4-b30c-1e40b473530f" (UID: "a995b7ae-83bc-44f4-b30c-1e40b473530f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:28:32.202838 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.202780    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-serving-cert" pod "a995b7ae-83bc-44f4-b30c-1e40b473530f" (UID: "a995b7ae-83bc-44f4-b30c-1e40b473530f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:28:32.202899 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.202853    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a995b7ae-83bc-44f4-b30c-1e40b473530f-kube-api-access-hhkmr" pod "a995b7ae-83bc-44f4-b30c-1e40b473530f" (UID: "a995b7ae-83bc-44f4-b30c-1e40b473530f"). InnerVolumeSpecName "kube-api-access-hhkmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:28:32.301894 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.301865    5051 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-oauth-config\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:32.301894 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.301893    5051 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-trusted-ca-bundle\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:32.302078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.301909    5051 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-config\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:32.302078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.301924    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhkmr\" (UniqueName: \"kubernetes.io/projected/a995b7ae-83bc-44f4-b30c-1e40b473530f-kube-api-access-hhkmr\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:32.302078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.301940    5051 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a995b7ae-83bc-44f4-b30c-1e40b473530f-console-serving-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:32.302078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.301955    5051 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:32.302078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.301969    5051 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a995b7ae-83bc-44f4-b30c-1e40b473530f-oauth-serving-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:32.302078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.301983    5051 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a995b7ae-83bc-44f4-b30c-1e40b473530f-tmp\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:28:32.766383 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:28:32.766348    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:28:32.893505 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.893475    5051 generic.go:356] "Generic (PLEG): container finished" podID="a995b7ae-83bc-44f4-b30c-1e40b473530f" containerID="bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c" exitCode=0
Nov 03 19:28:32.893841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.893527    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcc7b8c66-slr7n"
Nov 03 19:28:32.893841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.893565    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcc7b8c66-slr7n" event={"ID":"a995b7ae-83bc-44f4-b30c-1e40b473530f","Type":"ContainerDied","Data":"bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c"}
Nov 03 19:28:32.893841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.893620    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcc7b8c66-slr7n" event={"ID":"a995b7ae-83bc-44f4-b30c-1e40b473530f","Type":"ContainerDied","Data":"e3fa15df4dd796af64dcaff05ce5d213c10689cd67a5c54186ab43d96c4df2f2"}
Nov 03 19:28:32.893841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.893645    5051 scope.go:122] "RemoveContainer" containerID="bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c"
Nov 03 19:28:32.902167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.902110    5051 scope.go:122] "RemoveContainer" containerID="bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c"
Nov 03 19:28:32.902372 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:28:32.902352    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c\": container with ID starting with bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c not found: ID does not exist" containerID="bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c"
Nov 03 19:28:32.902422 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.902380    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c"} err="failed to get container status \"bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c\": rpc error: code = NotFound desc = could not find container \"bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c\": container with ID starting with bae70d932fe3ad3fe3e29499e04a91fde2e24290aadda901bfe1712a0ce56c6c not found: ID does not exist"
Nov 03 19:28:32.926951 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.926931    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fcc7b8c66-slr7n"]
Nov 03 19:28:32.928385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:32.928368    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fcc7b8c66-slr7n"]
Nov 03 19:28:33.226717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:28:33.226687    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a995b7ae-83bc-44f4-b30c-1e40b473530f" path="/var/lib/kubelet/pods/a995b7ae-83bc-44f4-b30c-1e40b473530f/volumes"
Nov 03 19:29:02.766220 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:29:02.766180    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:29:06.362749 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.362696    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-77c78ddc57-lbkk9"]
Nov 03 19:29:06.363180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.363147    5051 cpu_manager.go:414] "containerMap: removing container" podUID="a995b7ae-83bc-44f4-b30c-1e40b473530f" containerName="console"
Nov 03 19:29:06.363180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.363164    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a995b7ae-83bc-44f4-b30c-1e40b473530f" containerName="console"
Nov 03 19:29:06.363180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.363175    5051 cpu_manager.go:414] "containerMap: removing container" podUID="5f14faa0-564b-4e55-a3bc-984c152c5ee2" containerName="insights-gathering"
Nov 03 19:29:06.363180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.363181    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f14faa0-564b-4e55-a3bc-984c152c5ee2" containerName="insights-gathering"
Nov 03 19:29:06.363329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.363240    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="a995b7ae-83bc-44f4-b30c-1e40b473530f" containerName="console"
Nov 03 19:29:06.363329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.363247    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f14faa0-564b-4e55-a3bc-984c152c5ee2" containerName="insights-gathering"
Nov 03 19:29:06.366175 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.366147    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.377543 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.377516    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c78ddc57-lbkk9"]
Nov 03 19:29:06.471422 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.471387    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-service-ca\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.471422 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.471427    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-console-oauth-config\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.471636 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.471447    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6ngs\" (UniqueName: \"kubernetes.io/projected/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-kube-api-access-f6ngs\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.471636 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.471608    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-trusted-ca-bundle\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.471724 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.471650    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-oauth-serving-cert\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.471765 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.471717    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-console-serving-cert\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.471765 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.471738    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-console-config\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.471765 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.471756    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-tmp\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.573108 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.573064    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-trusted-ca-bundle\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.573303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.573127    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-oauth-serving-cert\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.573303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.573157    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-console-serving-cert\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.573303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.573178    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-console-config\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.573303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.573202    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-tmp\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.573303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.573260    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-service-ca\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.573544 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.573380    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-console-oauth-config\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.573544 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.573424    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6ngs\" (UniqueName: \"kubernetes.io/projected/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-kube-api-access-f6ngs\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.574019 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.573992    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-console-config\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.574236 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.574016    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-trusted-ca-bundle\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.574236 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.574073    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-service-ca\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.574377 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.574070    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-oauth-serving-cert\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.574377 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.574152    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-tmp\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.575713 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.575685    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-console-oauth-config\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.575879 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.575861    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-console-serving-cert\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.586018 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.585995    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6ngs\" (UniqueName: \"kubernetes.io/projected/81685b9f-a697-4a2d-94ba-b5364c9c6c1c-kube-api-access-f6ngs\") pod \"console-77c78ddc57-lbkk9\" (UID: \"81685b9f-a697-4a2d-94ba-b5364c9c6c1c\") " pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.680844 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.680753    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:06.956892 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:06.956864    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c78ddc57-lbkk9"]
Nov 03 19:29:06.958941 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:29:06.958912    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81685b9f_a697_4a2d_94ba_b5364c9c6c1c.slice/crio-1ff98a882f0263310e8e3f5352de3b9533dfbde0e3d8e69d48b6819f1323cda7 WatchSource:0}: Error finding container 1ff98a882f0263310e8e3f5352de3b9533dfbde0e3d8e69d48b6819f1323cda7: Status 404 returned error can't find the container with id 1ff98a882f0263310e8e3f5352de3b9533dfbde0e3d8e69d48b6819f1323cda7
Nov 03 19:29:07.031951 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:07.031921    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c78ddc57-lbkk9" event={"ID":"81685b9f-a697-4a2d-94ba-b5364c9c6c1c","Type":"ContainerStarted","Data":"a2de5b00edcae31c093ba104969368d09c43feeddbf8e56984f4b8a2b0b7f1b5"}
Nov 03 19:29:07.032188 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:07.031977    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c78ddc57-lbkk9" event={"ID":"81685b9f-a697-4a2d-94ba-b5364c9c6c1c","Type":"ContainerStarted","Data":"1ff98a882f0263310e8e3f5352de3b9533dfbde0e3d8e69d48b6819f1323cda7"}
Nov 03 19:29:07.053793 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:07.053743    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-77c78ddc57-lbkk9" podStartSLOduration=1.053729401 podStartE2EDuration="1.053729401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:29:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:29:07.053205476 +0000 UTC m=+147.961227436" watchObservedRunningTime="2026-11-03 19:29:07.053729401 +0000 UTC m=+147.961751333"
Nov 03 19:29:16.680961 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:16.680922    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:16.681462 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:16.680985    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:16.685473 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:16.685449    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:17.081843 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:17.081816    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77c78ddc57-lbkk9"
Nov 03 19:29:17.136004 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:17.135949    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fcdfcb4cc-hn578"]
Nov 03 19:29:28.126356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:28.126266    5051 generic.go:356] "Generic (PLEG): container finished" podID="474276d6-b0b6-4ac1-864a-1c57a6465bc0" containerID="f2920ee25c369f87061106c3de216598da717bb2ecc3b4d6972a790aea46a9d7" exitCode=0
Nov 03 19:29:28.126356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:28.126340    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-tddrc-xp7ln" event={"ID":"474276d6-b0b6-4ac1-864a-1c57a6465bc0","Type":"ContainerDied","Data":"f2920ee25c369f87061106c3de216598da717bb2ecc3b4d6972a790aea46a9d7"}
Nov 03 19:29:29.251190 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.251164    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:29:29.354764 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.354731    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/474276d6-b0b6-4ac1-864a-1c57a6465bc0-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/474276d6-b0b6-4ac1-864a-1c57a6465bc0-archives-path\") pod \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") "
Nov 03 19:29:29.354908 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.354776    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/474276d6-b0b6-4ac1-864a-1c57a6465bc0-kube-api-access-dppzb\" (UniqueName: \"kubernetes.io/projected/474276d6-b0b6-4ac1-864a-1c57a6465bc0-kube-api-access-dppzb\") pod \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") "
Nov 03 19:29:29.354908 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.354808    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-service-ca-bundle\") pod \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") "
Nov 03 19:29:29.354908 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.354846    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-trusted-ca-bundle\") pod \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\" (UID: \"474276d6-b0b6-4ac1-864a-1c57a6465bc0\") "
Nov 03 19:29:29.355128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.355100    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/474276d6-b0b6-4ac1-864a-1c57a6465bc0-archives-path" pod "474276d6-b0b6-4ac1-864a-1c57a6465bc0" (UID: "474276d6-b0b6-4ac1-864a-1c57a6465bc0"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:29:29.355228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.355207    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-service-ca-bundle" pod "474276d6-b0b6-4ac1-864a-1c57a6465bc0" (UID: "474276d6-b0b6-4ac1-864a-1c57a6465bc0"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:29:29.355318 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.355297    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-trusted-ca-bundle" pod "474276d6-b0b6-4ac1-864a-1c57a6465bc0" (UID: "474276d6-b0b6-4ac1-864a-1c57a6465bc0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:29:29.357155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.357128    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/474276d6-b0b6-4ac1-864a-1c57a6465bc0-kube-api-access-dppzb" pod "474276d6-b0b6-4ac1-864a-1c57a6465bc0" (UID: "474276d6-b0b6-4ac1-864a-1c57a6465bc0"). InnerVolumeSpecName "kube-api-access-dppzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:29:29.455941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.455875    5051 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/474276d6-b0b6-4ac1-864a-1c57a6465bc0-archives-path\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:29.455941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.455908    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dppzb\" (UniqueName: \"kubernetes.io/projected/474276d6-b0b6-4ac1-864a-1c57a6465bc0-kube-api-access-dppzb\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:29.455941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.455919    5051 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-service-ca-bundle\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:29.455941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:29.455927    5051 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474276d6-b0b6-4ac1-864a-1c57a6465bc0-trusted-ca-bundle\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:30.135523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:30.135498    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-tddrc-xp7ln"
Nov 03 19:29:30.135678 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:30.135485    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-tddrc-xp7ln" event={"ID":"474276d6-b0b6-4ac1-864a-1c57a6465bc0","Type":"ContainerDied","Data":"5401cca2317051740319f60d278ff4f337d10d29691a1c8d41705023f641954d"}
Nov 03 19:29:30.135678 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:30.135623    5051 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5401cca2317051740319f60d278ff4f337d10d29691a1c8d41705023f641954d"
Nov 03 19:29:32.765752 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:29:32.765716    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:29:42.162920 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.162873    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7fcdfcb4cc-hn578" podUID="d78e25ee-4672-4f09-b718-46a23c761ca4" containerName="console" containerID="cri-o://d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182" gracePeriod=15
Nov 03 19:29:42.419130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.419073    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:29:42.565780 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.565750    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d78e25ee-4672-4f09-b718-46a23c761ca4-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d78e25ee-4672-4f09-b718-46a23c761ca4-tmp\") pod \"d78e25ee-4672-4f09-b718-46a23c761ca4\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") "
Nov 03 19:29:42.565996 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.565804    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-serving-cert\") pod \"d78e25ee-4672-4f09-b718-46a23c761ca4\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") "
Nov 03 19:29:42.565996 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.565863    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d78e25ee-4672-4f09-b718-46a23c761ca4-kube-api-access-zhbmx\" (UniqueName: \"kubernetes.io/projected/d78e25ee-4672-4f09-b718-46a23c761ca4-kube-api-access-zhbmx\") pod \"d78e25ee-4672-4f09-b718-46a23c761ca4\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") "
Nov 03 19:29:42.565996 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.565937    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-oauth-config\") pod \"d78e25ee-4672-4f09-b718-46a23c761ca4\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") "
Nov 03 19:29:42.565996 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.565962    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-service-ca\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-service-ca\") pod \"d78e25ee-4672-4f09-b718-46a23c761ca4\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") "
Nov 03 19:29:42.566252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.565995    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-console-config\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-console-config\") pod \"d78e25ee-4672-4f09-b718-46a23c761ca4\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") "
Nov 03 19:29:42.566420 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.566108    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d78e25ee-4672-4f09-b718-46a23c761ca4-tmp" pod "d78e25ee-4672-4f09-b718-46a23c761ca4" (UID: "d78e25ee-4672-4f09-b718-46a23c761ca4"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:29:42.566519 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.566398    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-console-config" pod "d78e25ee-4672-4f09-b718-46a23c761ca4" (UID: "d78e25ee-4672-4f09-b718-46a23c761ca4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:29:42.566519 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.566411    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-trusted-ca-bundle\") pod \"d78e25ee-4672-4f09-b718-46a23c761ca4\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") "
Nov 03 19:29:42.566519 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.566464    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-service-ca" pod "d78e25ee-4672-4f09-b718-46a23c761ca4" (UID: "d78e25ee-4672-4f09-b718-46a23c761ca4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:29:42.566690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.566551    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-oauth-serving-cert\") pod \"d78e25ee-4672-4f09-b718-46a23c761ca4\" (UID: \"d78e25ee-4672-4f09-b718-46a23c761ca4\") "
Nov 03 19:29:42.566826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.566804    5051 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d78e25ee-4672-4f09-b718-46a23c761ca4-tmp\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:42.566826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.566827    5051 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:42.567074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.566827    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-trusted-ca-bundle" pod "d78e25ee-4672-4f09-b718-46a23c761ca4" (UID: "d78e25ee-4672-4f09-b718-46a23c761ca4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:29:42.567074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.566843    5051 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-console-config\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:42.567074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.567064    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-oauth-serving-cert" pod "d78e25ee-4672-4f09-b718-46a23c761ca4" (UID: "d78e25ee-4672-4f09-b718-46a23c761ca4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:29:42.568134 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.568106    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d78e25ee-4672-4f09-b718-46a23c761ca4-kube-api-access-zhbmx" pod "d78e25ee-4672-4f09-b718-46a23c761ca4" (UID: "d78e25ee-4672-4f09-b718-46a23c761ca4"). InnerVolumeSpecName "kube-api-access-zhbmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:29:42.568469 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.568441    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-serving-cert" pod "d78e25ee-4672-4f09-b718-46a23c761ca4" (UID: "d78e25ee-4672-4f09-b718-46a23c761ca4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:29:42.568469 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.568454    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-oauth-config" pod "d78e25ee-4672-4f09-b718-46a23c761ca4" (UID: "d78e25ee-4672-4f09-b718-46a23c761ca4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:29:42.668161 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.668123    5051 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-oauth-config\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:42.668161 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.668159    5051 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-trusted-ca-bundle\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:42.668161 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.668169    5051 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d78e25ee-4672-4f09-b718-46a23c761ca4-oauth-serving-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:42.668356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.668177    5051 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d78e25ee-4672-4f09-b718-46a23c761ca4-console-serving-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:42.668356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:42.668186    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhbmx\" (UniqueName: \"kubernetes.io/projected/d78e25ee-4672-4f09-b718-46a23c761ca4-kube-api-access-zhbmx\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:29:43.191214 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:43.191177    5051 generic.go:356] "Generic (PLEG): container finished" podID="d78e25ee-4672-4f09-b718-46a23c761ca4" containerID="d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182" exitCode=0
Nov 03 19:29:43.191619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:43.191232    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fcdfcb4cc-hn578"
Nov 03 19:29:43.191619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:43.191233    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fcdfcb4cc-hn578" event={"ID":"d78e25ee-4672-4f09-b718-46a23c761ca4","Type":"ContainerDied","Data":"d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182"}
Nov 03 19:29:43.191619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:43.191342    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fcdfcb4cc-hn578" event={"ID":"d78e25ee-4672-4f09-b718-46a23c761ca4","Type":"ContainerDied","Data":"6052e12b6b37b30b740204a7158f897bab5376cca040e55338ecc047cdc70711"}
Nov 03 19:29:43.191619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:43.191368    5051 scope.go:122] "RemoveContainer" containerID="d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182"
Nov 03 19:29:43.200007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:43.199989    5051 scope.go:122] "RemoveContainer" containerID="d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182"
Nov 03 19:29:43.200279 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:29:43.200261    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182\": container with ID starting with d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182 not found: ID does not exist" containerID="d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182"
Nov 03 19:29:43.200325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:43.200288    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182"} err="failed to get container status \"d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182\": rpc error: code = NotFound desc = could not find container \"d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182\": container with ID starting with d3cc89cb4cf9ad355f9232659573cd954a5b9d56a16349d336c010b681eb9182 not found: ID does not exist"
Nov 03 19:29:43.212985 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:43.212962    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fcdfcb4cc-hn578"]
Nov 03 19:29:43.216866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:43.216845    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7fcdfcb4cc-hn578"]
Nov 03 19:29:43.225595 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:29:43.225570    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d78e25ee-4672-4f09-b718-46a23c761ca4" path="/var/lib/kubelet/pods/d78e25ee-4672-4f09-b718-46a23c761ca4/volumes"
Nov 03 19:30:02.766325 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:30:02.766287    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:30:32.768808 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:30:32.768770    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:31:02.765831 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:31:02.765794    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:31:32.766297 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:31:32.766262    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:31:39.236264 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:31:39.236236    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:31:39.239091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:31:39.239069    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:32:02.767168 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:32:02.767131    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:32:32.765751 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:32:32.765701    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:33:02.765723 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:33:02.765679    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:33:04.658293 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.658259    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"]
Nov 03 19:33:04.658758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.658634    5051 cpu_manager.go:414] "containerMap: removing container" podUID="474276d6-b0b6-4ac1-864a-1c57a6465bc0" containerName="insights-gathering"
Nov 03 19:33:04.658758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.658649    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="474276d6-b0b6-4ac1-864a-1c57a6465bc0" containerName="insights-gathering"
Nov 03 19:33:04.658758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.658671    5051 cpu_manager.go:414] "containerMap: removing container" podUID="d78e25ee-4672-4f09-b718-46a23c761ca4" containerName="console"
Nov 03 19:33:04.658758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.658678    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d78e25ee-4672-4f09-b718-46a23c761ca4" containerName="console"
Nov 03 19:33:04.658758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.658752    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="d78e25ee-4672-4f09-b718-46a23c761ca4" containerName="console"
Nov 03 19:33:04.658911 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.658769    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="474276d6-b0b6-4ac1-864a-1c57a6465bc0" containerName="insights-gathering"
Nov 03 19:33:04.662033 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.662014    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.667609 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.667583    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 03 19:33:04.667768 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.667606    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 03 19:33:04.667768 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.667645    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 03 19:33:04.667956 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.667828    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-4m7l6\""
Nov 03 19:33:04.667956 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.667914    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 03 19:33:04.668214 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.668193    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 03 19:33:04.721781 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.721753    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"]
Nov 03 19:33:04.799713 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.799680    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-manifests\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.799865 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.799760    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktz4c\" (UniqueName: \"kubernetes.io/projected/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-kube-api-access-ktz4c\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.799865 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.799799    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-metrics-cert\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.799979 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.799912    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-webhook-cert\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.799979 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.799953    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.800095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.799989    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-tmp\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.900680 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.900647    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-manifests\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.900680 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.900685    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktz4c\" (UniqueName: \"kubernetes.io/projected/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-kube-api-access-ktz4c\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.900901 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.900708    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-metrics-cert\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.900901 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.900747    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-webhook-cert\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.900901 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.900772    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.901085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.900931    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-tmp\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.901146 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.901088    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-manifests\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.903142 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.903116    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-tmp\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.903399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.903376    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.903458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.903443    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-webhook-cert\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.903495 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.903457    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-metrics-cert\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.917298 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.917253    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktz4c\" (UniqueName: \"kubernetes.io/projected/cc111e19-92b3-4aea-b94f-b7fe3adb5f22-kube-api-access-ktz4c\") pod \"opendatahub-operator-controller-manager-7646b49899-mn6px\" (UID: \"cc111e19-92b3-4aea-b94f-b7fe3adb5f22\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:04.977423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:04.977395    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:05.275186 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:05.275162    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"]
Nov 03 19:33:05.278068 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:33:05.278015    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc111e19_92b3_4aea_b94f_b7fe3adb5f22.slice/crio-27b7fecb8caf67d4a44e8fcdef618a7a7ea5d42cbe4353cb8bbd1e7fe08cc3a4 WatchSource:0}: Error finding container 27b7fecb8caf67d4a44e8fcdef618a7a7ea5d42cbe4353cb8bbd1e7fe08cc3a4: Status 404 returned error can't find the container with id 27b7fecb8caf67d4a44e8fcdef618a7a7ea5d42cbe4353cb8bbd1e7fe08cc3a4
Nov 03 19:33:05.279749 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:05.279725    5051 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 19:33:06.000698 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:06.000658    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px" event={"ID":"cc111e19-92b3-4aea-b94f-b7fe3adb5f22","Type":"ContainerStarted","Data":"27b7fecb8caf67d4a44e8fcdef618a7a7ea5d42cbe4353cb8bbd1e7fe08cc3a4"}
Nov 03 19:33:09.016402 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:09.016370    5051 generic.go:356] "Generic (PLEG): container finished" podID="cc111e19-92b3-4aea-b94f-b7fe3adb5f22" containerID="5dc0f7c87eeb180e460f78e7b82d5131c78a9a15a4f0bc8a281e0599a496460d" exitCode=0
Nov 03 19:33:09.016799 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:09.016455    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px" event={"ID":"cc111e19-92b3-4aea-b94f-b7fe3adb5f22","Type":"ContainerDied","Data":"5dc0f7c87eeb180e460f78e7b82d5131c78a9a15a4f0bc8a281e0599a496460d"}
Nov 03 19:33:10.022740 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:10.022704    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px" event={"ID":"cc111e19-92b3-4aea-b94f-b7fe3adb5f22","Type":"ContainerStarted","Data":"e6161db50a1be0ad19424a7d3acb55dbbfc8db42097cbb72cda1dca4643db97d"}
Nov 03 19:33:10.023121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:10.022802    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:10.069847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:10.069788    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px" podStartSLOduration=2.349714765 podStartE2EDuration="6.069772685s" totalImagesPullingTime="3.316209349s" totalInitContainerRuntime="403.848571ms" isStatefulPod=false podCreationTimestamp="2026-11-03 19:33:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:33:10.068065356 +0000 UTC m=+390.976087309" watchObservedRunningTime="2026-11-03 19:33:10.069772685 +0000 UTC m=+390.977794625"
Nov 03 19:33:21.031812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:21.031729    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7646b49899-mn6px"
Nov 03 19:33:26.790245 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.790207    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"]
Nov 03 19:33:26.795485 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.795462    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"
Nov 03 19:33:26.801698 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.801679    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-zk7m7\""
Nov 03 19:33:26.805064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.805029    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 03 19:33:26.809621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.809602    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 03 19:33:26.811716 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.811694    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"]
Nov 03 19:33:26.868887 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.868856    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fffba9f9-cad3-4867-9e58-e7272e9b4303-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-4b5zv\" (UID: \"fffba9f9-cad3-4867-9e58-e7272e9b4303\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"
Nov 03 19:33:26.868984 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.868889    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr8dk\" (UniqueName: \"kubernetes.io/projected/fffba9f9-cad3-4867-9e58-e7272e9b4303-kube-api-access-mr8dk\") pod \"cert-manager-webhook-5dbbbf75cc-4b5zv\" (UID: \"fffba9f9-cad3-4867-9e58-e7272e9b4303\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"
Nov 03 19:33:26.970236 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.970210    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fffba9f9-cad3-4867-9e58-e7272e9b4303-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-4b5zv\" (UID: \"fffba9f9-cad3-4867-9e58-e7272e9b4303\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"
Nov 03 19:33:26.970329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.970243    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8dk\" (UniqueName: \"kubernetes.io/projected/fffba9f9-cad3-4867-9e58-e7272e9b4303-kube-api-access-mr8dk\") pod \"cert-manager-webhook-5dbbbf75cc-4b5zv\" (UID: \"fffba9f9-cad3-4867-9e58-e7272e9b4303\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"
Nov 03 19:33:26.986225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.986198    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fffba9f9-cad3-4867-9e58-e7272e9b4303-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-4b5zv\" (UID: \"fffba9f9-cad3-4867-9e58-e7272e9b4303\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"
Nov 03 19:33:26.986322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:26.986273    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr8dk\" (UniqueName: \"kubernetes.io/projected/fffba9f9-cad3-4867-9e58-e7272e9b4303-kube-api-access-mr8dk\") pod \"cert-manager-webhook-5dbbbf75cc-4b5zv\" (UID: \"fffba9f9-cad3-4867-9e58-e7272e9b4303\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"
Nov 03 19:33:27.119697 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:27.119634    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"
Nov 03 19:33:27.412851 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:33:27.412822    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfffba9f9_cad3_4867_9e58_e7272e9b4303.slice/crio-dde766109741c268aa8e95b7060c6aa789804faa68ace28a4e0d365fd8816995 WatchSource:0}: Error finding container dde766109741c268aa8e95b7060c6aa789804faa68ace28a4e0d365fd8816995: Status 404 returned error can't find the container with id dde766109741c268aa8e95b7060c6aa789804faa68ace28a4e0d365fd8816995
Nov 03 19:33:27.415839 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:27.415820    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"]
Nov 03 19:33:28.098894 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:28.098857    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv" event={"ID":"fffba9f9-cad3-4867-9e58-e7272e9b4303","Type":"ContainerStarted","Data":"dde766109741c268aa8e95b7060c6aa789804faa68ace28a4e0d365fd8816995"}
Nov 03 19:33:32.126598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:32.126561    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv" event={"ID":"fffba9f9-cad3-4867-9e58-e7272e9b4303","Type":"ContainerStarted","Data":"792caf852ca3fb0a2f1ae48487999108ccfa6c5bf4c4b85437e65975f521e06a"}
Nov 03 19:33:32.126970 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:32.126618    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"
Nov 03 19:33:32.194228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:32.194185    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv" podStartSLOduration=2.480201995 podStartE2EDuration="6.194148146s" totalImagesPullingTime="3.713946151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:33:32.192071843 +0000 UTC m=+413.100093779" watchObservedRunningTime="2026-11-03 19:33:32.194148146 +0000 UTC m=+413.102170084"
Nov 03 19:33:32.766627 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:33:32.766590    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:33:33.815549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:33.815510    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf"]
Nov 03 19:33:33.819059 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:33.819013    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf"
Nov 03 19:33:33.835465 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:33.835442    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf"]
Nov 03 19:33:33.860492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:33.860474    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7tfwk\""
Nov 03 19:33:33.937648 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:33.937624    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa5f4c8d-09d0-4543-a0db-b88a238ffa03-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-v79wf\" (UID: \"fa5f4c8d-09d0-4543-a0db-b88a238ffa03\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf"
Nov 03 19:33:33.937762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:33.937684    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqz67\" (UniqueName: \"kubernetes.io/projected/fa5f4c8d-09d0-4543-a0db-b88a238ffa03-kube-api-access-tqz67\") pod \"cert-manager-cainjector-5c57bbd4fc-v79wf\" (UID: \"fa5f4c8d-09d0-4543-a0db-b88a238ffa03\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf"
Nov 03 19:33:34.039034 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.039001    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa5f4c8d-09d0-4543-a0db-b88a238ffa03-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-v79wf\" (UID: \"fa5f4c8d-09d0-4543-a0db-b88a238ffa03\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf"
Nov 03 19:33:34.039188 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.039114    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqz67\" (UniqueName: \"kubernetes.io/projected/fa5f4c8d-09d0-4543-a0db-b88a238ffa03-kube-api-access-tqz67\") pod \"cert-manager-cainjector-5c57bbd4fc-v79wf\" (UID: \"fa5f4c8d-09d0-4543-a0db-b88a238ffa03\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf"
Nov 03 19:33:34.058388 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.058356    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa5f4c8d-09d0-4543-a0db-b88a238ffa03-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-v79wf\" (UID: \"fa5f4c8d-09d0-4543-a0db-b88a238ffa03\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf"
Nov 03 19:33:34.058504 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.058428    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqz67\" (UniqueName: \"kubernetes.io/projected/fa5f4c8d-09d0-4543-a0db-b88a238ffa03-kube-api-access-tqz67\") pod \"cert-manager-cainjector-5c57bbd4fc-v79wf\" (UID: \"fa5f4c8d-09d0-4543-a0db-b88a238ffa03\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf"
Nov 03 19:33:34.130755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.130687    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf"
Nov 03 19:33:34.412763 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.412735    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf"]
Nov 03 19:33:34.415031 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:33:34.415005    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa5f4c8d_09d0_4543_a0db_b88a238ffa03.slice/crio-c90767769e74e2a05f35f082e5833f56e115b0369f3e7cc213e6a2f1b50bb8c1 WatchSource:0}: Error finding container c90767769e74e2a05f35f082e5833f56e115b0369f3e7cc213e6a2f1b50bb8c1: Status 404 returned error can't find the container with id c90767769e74e2a05f35f082e5833f56e115b0369f3e7cc213e6a2f1b50bb8c1
Nov 03 19:33:34.575132 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.575094    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl"]
Nov 03 19:33:34.578480 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.578458    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl"
Nov 03 19:33:34.587869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.587843    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-sj9mt\""
Nov 03 19:33:34.588858 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.588840    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 03 19:33:34.588984 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.588857    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 03 19:33:34.594617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.594590    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl"]
Nov 03 19:33:34.643468 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.643436    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc80a989-407b-4129-8954-a9d3c8249aae-tmp\") pod \"jobset-operator-674cd59dc5-rlbrl\" (UID: \"dc80a989-407b-4129-8954-a9d3c8249aae\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl"
Nov 03 19:33:34.643468 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.643474    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dhfg\" (UniqueName: \"kubernetes.io/projected/dc80a989-407b-4129-8954-a9d3c8249aae-kube-api-access-9dhfg\") pod \"jobset-operator-674cd59dc5-rlbrl\" (UID: \"dc80a989-407b-4129-8954-a9d3c8249aae\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl"
Nov 03 19:33:34.744019 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.743986    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc80a989-407b-4129-8954-a9d3c8249aae-tmp\") pod \"jobset-operator-674cd59dc5-rlbrl\" (UID: \"dc80a989-407b-4129-8954-a9d3c8249aae\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl"
Nov 03 19:33:34.744019 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.744025    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dhfg\" (UniqueName: \"kubernetes.io/projected/dc80a989-407b-4129-8954-a9d3c8249aae-kube-api-access-9dhfg\") pod \"jobset-operator-674cd59dc5-rlbrl\" (UID: \"dc80a989-407b-4129-8954-a9d3c8249aae\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl"
Nov 03 19:33:34.744456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.744432    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc80a989-407b-4129-8954-a9d3c8249aae-tmp\") pod \"jobset-operator-674cd59dc5-rlbrl\" (UID: \"dc80a989-407b-4129-8954-a9d3c8249aae\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl"
Nov 03 19:33:34.758841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.758814    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dhfg\" (UniqueName: \"kubernetes.io/projected/dc80a989-407b-4129-8954-a9d3c8249aae-kube-api-access-9dhfg\") pod \"jobset-operator-674cd59dc5-rlbrl\" (UID: \"dc80a989-407b-4129-8954-a9d3c8249aae\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl"
Nov 03 19:33:34.889690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:34.889656    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl"
Nov 03 19:33:35.140387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:35.140306    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf" event={"ID":"fa5f4c8d-09d0-4543-a0db-b88a238ffa03","Type":"ContainerStarted","Data":"67528e36e96a800642ff7cbe914d6887d0b31928a7d3af8c68b2faee56b4c450"}
Nov 03 19:33:35.140387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:35.140351    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf" event={"ID":"fa5f4c8d-09d0-4543-a0db-b88a238ffa03","Type":"ContainerStarted","Data":"c90767769e74e2a05f35f082e5833f56e115b0369f3e7cc213e6a2f1b50bb8c1"}
Nov 03 19:33:35.174783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:35.174736    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-v79wf" podStartSLOduration=2.174722058 podStartE2EDuration="2.174722058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:33:35.173997692 +0000 UTC m=+416.082019632" watchObservedRunningTime="2026-11-03 19:33:35.174722058 +0000 UTC m=+416.082743996"
Nov 03 19:33:35.183083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:35.183057    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl"]
Nov 03 19:33:35.185357 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:33:35.185332    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc80a989_407b_4129_8954_a9d3c8249aae.slice/crio-3049c5d46df108964b11fef6408a0719d732ecd5dcd89a80db90c07566d316fe WatchSource:0}: Error finding container 3049c5d46df108964b11fef6408a0719d732ecd5dcd89a80db90c07566d316fe: Status 404 returned error can't find the container with id 3049c5d46df108964b11fef6408a0719d732ecd5dcd89a80db90c07566d316fe
Nov 03 19:33:36.146670 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.146629    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl" event={"ID":"dc80a989-407b-4129-8954-a9d3c8249aae","Type":"ContainerStarted","Data":"3049c5d46df108964b11fef6408a0719d732ecd5dcd89a80db90c07566d316fe"}
Nov 03 19:33:36.692383 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.692352    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"]
Nov 03 19:33:36.696297 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.696275    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"
Nov 03 19:33:36.716004 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.715981    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"]
Nov 03 19:33:36.729109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.729085    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 03 19:33:36.729647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.729625    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 03 19:33:36.730556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.730536    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-86cdm\""
Nov 03 19:33:36.764101 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.764076    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/292c8c65-8eb5-45ce-b09b-6b235e2e2c9c-tmp\") pod \"kube-auth-proxy-7b4fd54d5-44frg\" (UID: \"292c8c65-8eb5-45ce-b09b-6b235e2e2c9c\") " pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"
Nov 03 19:33:36.764220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.764142    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/292c8c65-8eb5-45ce-b09b-6b235e2e2c9c-tls-certs\") pod \"kube-auth-proxy-7b4fd54d5-44frg\" (UID: \"292c8c65-8eb5-45ce-b09b-6b235e2e2c9c\") " pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"
Nov 03 19:33:36.764266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.764243    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg5ks\" (UniqueName: \"kubernetes.io/projected/292c8c65-8eb5-45ce-b09b-6b235e2e2c9c-kube-api-access-lg5ks\") pod \"kube-auth-proxy-7b4fd54d5-44frg\" (UID: \"292c8c65-8eb5-45ce-b09b-6b235e2e2c9c\") " pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"
Nov 03 19:33:36.865334 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.865301    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg5ks\" (UniqueName: \"kubernetes.io/projected/292c8c65-8eb5-45ce-b09b-6b235e2e2c9c-kube-api-access-lg5ks\") pod \"kube-auth-proxy-7b4fd54d5-44frg\" (UID: \"292c8c65-8eb5-45ce-b09b-6b235e2e2c9c\") " pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"
Nov 03 19:33:36.865514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.865372    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/292c8c65-8eb5-45ce-b09b-6b235e2e2c9c-tmp\") pod \"kube-auth-proxy-7b4fd54d5-44frg\" (UID: \"292c8c65-8eb5-45ce-b09b-6b235e2e2c9c\") " pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"
Nov 03 19:33:36.865514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.865455    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/292c8c65-8eb5-45ce-b09b-6b235e2e2c9c-tls-certs\") pod \"kube-auth-proxy-7b4fd54d5-44frg\" (UID: \"292c8c65-8eb5-45ce-b09b-6b235e2e2c9c\") " pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"
Nov 03 19:33:36.867708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.867684    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/292c8c65-8eb5-45ce-b09b-6b235e2e2c9c-tmp\") pod \"kube-auth-proxy-7b4fd54d5-44frg\" (UID: \"292c8c65-8eb5-45ce-b09b-6b235e2e2c9c\") " pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"
Nov 03 19:33:36.867922 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.867900    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/292c8c65-8eb5-45ce-b09b-6b235e2e2c9c-tls-certs\") pod \"kube-auth-proxy-7b4fd54d5-44frg\" (UID: \"292c8c65-8eb5-45ce-b09b-6b235e2e2c9c\") " pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"
Nov 03 19:33:36.898815 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:36.898788    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg5ks\" (UniqueName: \"kubernetes.io/projected/292c8c65-8eb5-45ce-b09b-6b235e2e2c9c-kube-api-access-lg5ks\") pod \"kube-auth-proxy-7b4fd54d5-44frg\" (UID: \"292c8c65-8eb5-45ce-b09b-6b235e2e2c9c\") " pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"
Nov 03 19:33:37.011881 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:37.011847    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"
Nov 03 19:33:37.350856 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:33:37.350820    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod292c8c65_8eb5_45ce_b09b_6b235e2e2c9c.slice/crio-2a76131f193f5f078042562f040dbac36ed52bb1b1f672458cbf2af48b5d1fbb WatchSource:0}: Error finding container 2a76131f193f5f078042562f040dbac36ed52bb1b1f672458cbf2af48b5d1fbb: Status 404 returned error can't find the container with id 2a76131f193f5f078042562f040dbac36ed52bb1b1f672458cbf2af48b5d1fbb
Nov 03 19:33:37.351418 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:37.351390    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg"]
Nov 03 19:33:38.133225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:38.133196    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv"
Nov 03 19:33:38.170515 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:38.170471    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg" event={"ID":"292c8c65-8eb5-45ce-b09b-6b235e2e2c9c","Type":"ContainerStarted","Data":"2a76131f193f5f078042562f040dbac36ed52bb1b1f672458cbf2af48b5d1fbb"}
Nov 03 19:33:40.167942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:40.167919    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 03 19:33:40.186113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:40.186081    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl" event={"ID":"dc80a989-407b-4129-8954-a9d3c8249aae","Type":"ContainerStarted","Data":"0a742eeae7c70a7776c8572e80e21f80c21aa6fa49a877c504b2a25b48bfc90a"}
Nov 03 19:33:40.219808 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:40.219766    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rlbrl" podStartSLOduration=1.445795017 podStartE2EDuration="6.219753239s" totalImagesPullingTime="4.773958222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:33:40.21773721 +0000 UTC m=+421.125759153" watchObservedRunningTime="2026-11-03 19:33:40.219753239 +0000 UTC m=+421.127775177"
Nov 03 19:33:41.191850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:41.191811    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg" event={"ID":"292c8c65-8eb5-45ce-b09b-6b235e2e2c9c","Type":"ContainerStarted","Data":"b0159df6de6f80e2822e8f81e77a2ca6388ed159b6d57719c494573fefbef1b5"}
Nov 03 19:33:41.220791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:41.217936    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b4fd54d5-44frg" podStartSLOduration=2.410837394 podStartE2EDuration="5.217918806s" totalImagesPullingTime="2.807081412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:33:41.214627121 +0000 UTC m=+422.122649056" watchObservedRunningTime="2026-11-03 19:33:41.217918806 +0000 UTC m=+422.125940748"
Nov 03 19:33:42.408238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.408205    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:33:42.411826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.411809    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"
Nov 03 19:33:42.424784 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.424749    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 03 19:33:42.424919 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.424879    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 03 19:33:42.424919 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.424886    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:33:42.425034 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.424892    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 03 19:33:42.433054 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.433025    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-frwwl\""
Nov 03 19:33:42.459026 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.459004    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:33:42.511456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.511412    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t27cq\" (UniqueName: \"kubernetes.io/projected/77139ff2-796c-4caa-bd8b-e78bc69db3e6-kube-api-access-t27cq\") pod \"limitador-limitador-7c9b87d574-2cwgg\" (UID: \"77139ff2-796c-4caa-bd8b-e78bc69db3e6\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"
Nov 03 19:33:42.511734 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.511704    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77139ff2-796c-4caa-bd8b-e78bc69db3e6-config-file\") pod \"limitador-limitador-7c9b87d574-2cwgg\" (UID: \"77139ff2-796c-4caa-bd8b-e78bc69db3e6\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"
Nov 03 19:33:42.613244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.613213    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t27cq\" (UniqueName: \"kubernetes.io/projected/77139ff2-796c-4caa-bd8b-e78bc69db3e6-kube-api-access-t27cq\") pod \"limitador-limitador-7c9b87d574-2cwgg\" (UID: \"77139ff2-796c-4caa-bd8b-e78bc69db3e6\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"
Nov 03 19:33:42.613416 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.613252    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77139ff2-796c-4caa-bd8b-e78bc69db3e6-config-file\") pod \"limitador-limitador-7c9b87d574-2cwgg\" (UID: \"77139ff2-796c-4caa-bd8b-e78bc69db3e6\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"
Nov 03 19:33:42.613920 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.613900    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77139ff2-796c-4caa-bd8b-e78bc69db3e6-config-file\") pod \"limitador-limitador-7c9b87d574-2cwgg\" (UID: \"77139ff2-796c-4caa-bd8b-e78bc69db3e6\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"
Nov 03 19:33:42.625985 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.625961    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t27cq\" (UniqueName: \"kubernetes.io/projected/77139ff2-796c-4caa-bd8b-e78bc69db3e6-kube-api-access-t27cq\") pod \"limitador-limitador-7c9b87d574-2cwgg\" (UID: \"77139ff2-796c-4caa-bd8b-e78bc69db3e6\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"
Nov 03 19:33:42.725062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:42.725016    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"
Nov 03 19:33:43.002930 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:43.002900    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:33:43.005422 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:33:43.005395    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77139ff2_796c_4caa_bd8b_e78bc69db3e6.slice/crio-51db40099f4eb93d014c1f101a8282050a7ff6ce08ad0b1d6d3d7e23bff12211 WatchSource:0}: Error finding container 51db40099f4eb93d014c1f101a8282050a7ff6ce08ad0b1d6d3d7e23bff12211: Status 404 returned error can't find the container with id 51db40099f4eb93d014c1f101a8282050a7ff6ce08ad0b1d6d3d7e23bff12211
Nov 03 19:33:43.201555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:43.201519    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg" event={"ID":"77139ff2-796c-4caa-bd8b-e78bc69db3e6","Type":"ContainerStarted","Data":"51db40099f4eb93d014c1f101a8282050a7ff6ce08ad0b1d6d3d7e23bff12211"}
Nov 03 19:33:46.218642 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:46.218604    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg" event={"ID":"77139ff2-796c-4caa-bd8b-e78bc69db3e6","Type":"ContainerStarted","Data":"62d7bd4431eaf4162a2d9a3d80addc4f245dfd99678f307709b07ae523fd9e0c"}
Nov 03 19:33:46.219148 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:46.218737    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"
Nov 03 19:33:46.266611 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:46.266559    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg" podStartSLOduration=1.799289881 podStartE2EDuration="4.266546443s" totalImagesPullingTime="2.467256562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:33:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:33:46.250583389 +0000 UTC m=+427.158605347" watchObservedRunningTime="2026-11-03 19:33:46.266546443 +0000 UTC m=+427.174568421"
Nov 03 19:33:57.226660 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:33:57.226634    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"
Nov 03 19:34:02.766702 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:34:02.766667    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:34:32.765861 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:34:32.765824    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:34:36.622192 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.622158    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-z87sc"]
Nov 03 19:34:36.624775 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.624754    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc"
Nov 03 19:34:36.637398 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.637377    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Nov 03 19:34:36.640439 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.640422    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-fkpzs\""
Nov 03 19:34:36.649257 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.649236    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 03 19:34:36.649257 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.649235    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 03 19:34:36.666907 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.666883    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-z87sc"]
Nov 03 19:34:36.716599 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.716569    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-vnbhz"]
Nov 03 19:34:36.718844 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.718827    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:36.746840 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.746811    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-vnbhz"]
Nov 03 19:34:36.771627 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.771571    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/812da5c0-5a4f-4013-b799-bf5e410f49a2-cert\") pod \"kserve-controller-manager-776cc66d47-z87sc\" (UID: \"812da5c0-5a4f-4013-b799-bf5e410f49a2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc"
Nov 03 19:34:36.771778 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.771669    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/745c0adf-f3a5-4eb7-adfd-c1d4ebb72806-cert\") pod \"odh-model-controller-769875db56-vnbhz\" (UID: \"745c0adf-f3a5-4eb7-adfd-c1d4ebb72806\") " pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:36.771778 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.771704    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxd4n\" (UniqueName: \"kubernetes.io/projected/745c0adf-f3a5-4eb7-adfd-c1d4ebb72806-kube-api-access-nxd4n\") pod \"odh-model-controller-769875db56-vnbhz\" (UID: \"745c0adf-f3a5-4eb7-adfd-c1d4ebb72806\") " pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:36.771778 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.771775    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcprn\" (UniqueName: \"kubernetes.io/projected/812da5c0-5a4f-4013-b799-bf5e410f49a2-kube-api-access-pcprn\") pod \"kserve-controller-manager-776cc66d47-z87sc\" (UID: \"812da5c0-5a4f-4013-b799-bf5e410f49a2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc"
Nov 03 19:34:36.771908 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.771822    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745c0adf-f3a5-4eb7-adfd-c1d4ebb72806-metrics-certs\") pod \"odh-model-controller-769875db56-vnbhz\" (UID: \"745c0adf-f3a5-4eb7-adfd-c1d4ebb72806\") " pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:36.821926 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.821901    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Nov 03 19:34:36.821926 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.821916    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Nov 03 19:34:36.821926 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.821924    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-862c4\""
Nov 03 19:34:36.872365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.872300    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745c0adf-f3a5-4eb7-adfd-c1d4ebb72806-metrics-certs\") pod \"odh-model-controller-769875db56-vnbhz\" (UID: \"745c0adf-f3a5-4eb7-adfd-c1d4ebb72806\") " pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:36.872365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.872361    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/812da5c0-5a4f-4013-b799-bf5e410f49a2-cert\") pod \"kserve-controller-manager-776cc66d47-z87sc\" (UID: \"812da5c0-5a4f-4013-b799-bf5e410f49a2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc"
Nov 03 19:34:36.872520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.872390    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/745c0adf-f3a5-4eb7-adfd-c1d4ebb72806-cert\") pod \"odh-model-controller-769875db56-vnbhz\" (UID: \"745c0adf-f3a5-4eb7-adfd-c1d4ebb72806\") " pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:36.872520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.872413    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxd4n\" (UniqueName: \"kubernetes.io/projected/745c0adf-f3a5-4eb7-adfd-c1d4ebb72806-kube-api-access-nxd4n\") pod \"odh-model-controller-769875db56-vnbhz\" (UID: \"745c0adf-f3a5-4eb7-adfd-c1d4ebb72806\") " pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:36.872520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.872466    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcprn\" (UniqueName: \"kubernetes.io/projected/812da5c0-5a4f-4013-b799-bf5e410f49a2-kube-api-access-pcprn\") pod \"kserve-controller-manager-776cc66d47-z87sc\" (UID: \"812da5c0-5a4f-4013-b799-bf5e410f49a2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc"
Nov 03 19:34:36.874952 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.874933    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/812da5c0-5a4f-4013-b799-bf5e410f49a2-cert\") pod \"kserve-controller-manager-776cc66d47-z87sc\" (UID: \"812da5c0-5a4f-4013-b799-bf5e410f49a2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc"
Nov 03 19:34:36.875133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.875112    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/745c0adf-f3a5-4eb7-adfd-c1d4ebb72806-cert\") pod \"odh-model-controller-769875db56-vnbhz\" (UID: \"745c0adf-f3a5-4eb7-adfd-c1d4ebb72806\") " pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:36.875181 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.875131    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745c0adf-f3a5-4eb7-adfd-c1d4ebb72806-metrics-certs\") pod \"odh-model-controller-769875db56-vnbhz\" (UID: \"745c0adf-f3a5-4eb7-adfd-c1d4ebb72806\") " pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:36.891466 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.891446    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxd4n\" (UniqueName: \"kubernetes.io/projected/745c0adf-f3a5-4eb7-adfd-c1d4ebb72806-kube-api-access-nxd4n\") pod \"odh-model-controller-769875db56-vnbhz\" (UID: \"745c0adf-f3a5-4eb7-adfd-c1d4ebb72806\") " pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:36.893155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.893140    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcprn\" (UniqueName: \"kubernetes.io/projected/812da5c0-5a4f-4013-b799-bf5e410f49a2-kube-api-access-pcprn\") pod \"kserve-controller-manager-776cc66d47-z87sc\" (UID: \"812da5c0-5a4f-4013-b799-bf5e410f49a2\") " pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc"
Nov 03 19:34:36.937130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:36.937110    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc"
Nov 03 19:34:37.030328 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:37.030300    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:37.218426 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:37.218398    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-z87sc"]
Nov 03 19:34:37.220419 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:34:37.220393    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812da5c0_5a4f_4013_b799_bf5e410f49a2.slice/crio-652dcfce64828e8d2b19a48fcbf5380a55f1386b932efcb7be335abf44857720 WatchSource:0}: Error finding container 652dcfce64828e8d2b19a48fcbf5380a55f1386b932efcb7be335abf44857720: Status 404 returned error can't find the container with id 652dcfce64828e8d2b19a48fcbf5380a55f1386b932efcb7be335abf44857720
Nov 03 19:34:37.375987 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:37.375959    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-vnbhz"]
Nov 03 19:34:37.377064 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:34:37.377018    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod745c0adf_f3a5_4eb7_adfd_c1d4ebb72806.slice/crio-5f1c4fee7d7b480ed133a15f4557bbd386b702a1bb5c279f8c0a1ed477ed5248 WatchSource:0}: Error finding container 5f1c4fee7d7b480ed133a15f4557bbd386b702a1bb5c279f8c0a1ed477ed5248: Status 404 returned error can't find the container with id 5f1c4fee7d7b480ed133a15f4557bbd386b702a1bb5c279f8c0a1ed477ed5248
Nov 03 19:34:37.436757 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:37.436729    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc" event={"ID":"812da5c0-5a4f-4013-b799-bf5e410f49a2","Type":"ContainerStarted","Data":"652dcfce64828e8d2b19a48fcbf5380a55f1386b932efcb7be335abf44857720"}
Nov 03 19:34:37.437850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:37.437821    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-vnbhz" event={"ID":"745c0adf-f3a5-4eb7-adfd-c1d4ebb72806","Type":"ContainerStarted","Data":"5f1c4fee7d7b480ed133a15f4557bbd386b702a1bb5c279f8c0a1ed477ed5248"}
Nov 03 19:34:40.456498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:40.456408    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc" event={"ID":"812da5c0-5a4f-4013-b799-bf5e410f49a2","Type":"ContainerStarted","Data":"5d03df4889a0383ca8daca669e0f1273331c28a9c343912cccdc3eb508c97d24"}
Nov 03 19:34:40.456993 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:40.456513    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc"
Nov 03 19:34:40.458523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:40.458492    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-vnbhz" event={"ID":"745c0adf-f3a5-4eb7-adfd-c1d4ebb72806","Type":"ContainerStarted","Data":"a49276c29095604b5a96fd8824c56b7763a8a47e064f654866ce18d92326bf4f"}
Nov 03 19:34:40.458665 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:40.458598    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:34:40.494576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:40.494522    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc" podStartSLOduration=2.076173255 podStartE2EDuration="4.494509211s" totalImagesPullingTime="2.418335956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:34:40.492424617 +0000 UTC m=+481.400446558" watchObservedRunningTime="2026-11-03 19:34:40.494509211 +0000 UTC m=+481.402531173"
Nov 03 19:34:40.535697 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:40.535654    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-vnbhz" podStartSLOduration=1.765600966 podStartE2EDuration="4.535643813s" totalImagesPullingTime="2.770042847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:34:40.534658406 +0000 UTC m=+481.442680345" watchObservedRunningTime="2026-11-03 19:34:40.535643813 +0000 UTC m=+481.443665751"
Nov 03 19:34:51.465061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:34:51.464969    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-vnbhz"
Nov 03 19:35:02.766381 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:35:02.766347    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:35:02.931193 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:02.928206    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:35:11.468261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:11.468231    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-z87sc"
Nov 03 19:35:12.687711 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.687670    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"]
Nov 03 19:35:12.692546 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.692521    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.705824 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.705803    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 19:35:12.705933 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.705892    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 19:35:12.709848 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.709824    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"]
Nov 03 19:35:12.745888 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.745861    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-wm9nt\""
Nov 03 19:35:12.746075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.745944    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 03 19:35:12.746075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.745996    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 03 19:35:12.768025 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.768003    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgc9r\" (UniqueName: \"kubernetes.io/projected/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-kube-api-access-jgc9r\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.768147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.768074    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-metrics-certs\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.768210 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.768145    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-openshift-service-ca\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.768210 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.768197    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-maas-api-tls\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.868747 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.868715    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgc9r\" (UniqueName: \"kubernetes.io/projected/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-kube-api-access-jgc9r\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.868941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.868770    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-metrics-certs\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.868941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.868789    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-openshift-service-ca\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.868941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.868820    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-maas-api-tls\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.869582 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.869563    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-openshift-service-ca\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.871322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.871300    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-maas-api-tls\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.871413 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.871331    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-metrics-certs\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:12.881808 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:12.881785    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgc9r\" (UniqueName: \"kubernetes.io/projected/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-kube-api-access-jgc9r\") pod \"maas-api-57c44bc9cc-rzrqz\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") " pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:13.004271 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:13.004244    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:13.275934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:13.275908    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"]
Nov 03 19:35:13.277326 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:35:13.277295    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ecdc8b_d2bc_4019_86d8_9a0847d72cfb.slice/crio-1f6e506b9aeee49a2d987248a0768f8ab579911fc43b569005567a5df279a85a WatchSource:0}: Error finding container 1f6e506b9aeee49a2d987248a0768f8ab579911fc43b569005567a5df279a85a: Status 404 returned error can't find the container with id 1f6e506b9aeee49a2d987248a0768f8ab579911fc43b569005567a5df279a85a
Nov 03 19:35:13.611225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:13.611148    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz" event={"ID":"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb","Type":"ContainerStarted","Data":"1f6e506b9aeee49a2d987248a0768f8ab579911fc43b569005567a5df279a85a"}
Nov 03 19:35:15.621367 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:15.621340    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz" event={"ID":"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb","Type":"ContainerStarted","Data":"c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc"}
Nov 03 19:35:15.621651 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:15.621471    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:15.642721 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:15.642676    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz" podStartSLOduration=1.360839635 podStartE2EDuration="3.642663486s" totalImagesPullingTime="2.281823851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:35:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:35:15.64199108 +0000 UTC m=+516.550013046" watchObservedRunningTime="2026-11-03 19:35:15.642663486 +0000 UTC m=+516.550685424"
Nov 03 19:35:21.630899 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:35:21.630872    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 19:35:32.766698 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:35:32.766661    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:36:02.765744 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:36:02.765705    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:36:27.583327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.583253    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"]
Nov 03 19:36:27.586547 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.586529    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.594726 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.594702    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bfhff\""
Nov 03 19:36:27.594885 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.594868    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 03 19:36:27.594969 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.594897    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Nov 03 19:36:27.594969 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.594914    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 03 19:36:27.601458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.601438    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"]
Nov 03 19:36:27.689029 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.688998    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.689179 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.689034    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.689179 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.689077    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8f9579a-928c-41e0-a4ad-3438618bae19-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.689249 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.689206    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6gz5\" (UniqueName: \"kubernetes.io/projected/c8f9579a-928c-41e0-a4ad-3438618bae19-kube-api-access-k6gz5\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.689249 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.689242    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.689343 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.689329    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.790427 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.790401    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6gz5\" (UniqueName: \"kubernetes.io/projected/c8f9579a-928c-41e0-a4ad-3438618bae19-kube-api-access-k6gz5\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.790551 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.790434    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.790551 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.790468    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.790551 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.790494    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.790551 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.790526    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.790551 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.790547    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8f9579a-928c-41e0-a4ad-3438618bae19-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.790920 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.790893    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.790920 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.790912    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.791037 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.790941    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.792842 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.792819    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8f9579a-928c-41e0-a4ad-3438618bae19-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.792989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.792973    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8f9579a-928c-41e0-a4ad-3438618bae19-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.802895 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.802876    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6gz5\" (UniqueName: \"kubernetes.io/projected/c8f9579a-928c-41e0-a4ad-3438618bae19-kube-api-access-k6gz5\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n\" (UID: \"c8f9579a-928c-41e0-a4ad-3438618bae19\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:27.899088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:27.899019    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:28.110035 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:28.108636    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:36:28.170342 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:28.170318    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"]
Nov 03 19:36:28.172244 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:36:28.172220    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8f9579a_928c_41e0_a4ad_3438618bae19.slice/crio-88f7b1d57ff6e9ef271a64abf6db4e3b636c021646d1e5ef6da91eaf8ee766ac WatchSource:0}: Error finding container 88f7b1d57ff6e9ef271a64abf6db4e3b636c021646d1e5ef6da91eaf8ee766ac: Status 404 returned error can't find the container with id 88f7b1d57ff6e9ef271a64abf6db4e3b636c021646d1e5ef6da91eaf8ee766ac
Nov 03 19:36:28.968348 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:28.968305    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n" event={"ID":"c8f9579a-928c-41e0-a4ad-3438618bae19","Type":"ContainerStarted","Data":"88f7b1d57ff6e9ef271a64abf6db4e3b636c021646d1e5ef6da91eaf8ee766ac"}
Nov 03 19:36:29.974252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:29.974218    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n" event={"ID":"c8f9579a-928c-41e0-a4ad-3438618bae19","Type":"ContainerStarted","Data":"ce0c9394354caba757d1db1fa3013d54338165e1d7a0c5f0e6609ffc0cb22c32"}
Nov 03 19:36:29.996847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:29.996803    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n" podStartSLOduration=1.5888424909999999 podStartE2EDuration="2.996790229s" totalImagesPullingTime="1.407947738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:36:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:36:29.99542911 +0000 UTC m=+590.903451060" watchObservedRunningTime="2026-11-03 19:36:29.996790229 +0000 UTC m=+590.904812168"
Nov 03 19:36:30.346749 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:30.346667    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:36:30.900147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:30.900115    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:30.972820 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:30.972781    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"]
Nov 03 19:36:30.976399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:30.976370    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:30.987962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:30.987937    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"]
Nov 03 19:36:30.988268 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:30.988243    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Nov 03 19:36:31.121403 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.121373    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsm7b\" (UniqueName: \"kubernetes.io/projected/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-kube-api-access-bsm7b\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.121557 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.121417    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.121557 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.121510    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.121650 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.121633    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.121702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.121689    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.121739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.121717    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.222382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.222346    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.222565 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.222440    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.222565 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.222478    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.222565 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.222514    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.222565 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.222543    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsm7b\" (UniqueName: \"kubernetes.io/projected/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-kube-api-access-bsm7b\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.222793 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.222582    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.222875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.222851    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.223002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.222974    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.223140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.223067    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.225235 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.225215    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.226364 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.226330    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.234922 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.234898    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsm7b\" (UniqueName: \"kubernetes.io/projected/e4d4d3cd-6773-4b3b-80c6-52f5f9af92da-kube-api-access-bsm7b\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-v4fqs\" (UID: \"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.292086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.292055    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:31.583774 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.583749    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"]
Nov 03 19:36:31.585631 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:36:31.585603    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d4d3cd_6773_4b3b_80c6_52f5f9af92da.slice/crio-46ca0084bb7b63755fd1562ddb0f26ee32a7a5c107d9209638cd8c150af3b1cc WatchSource:0}: Error finding container 46ca0084bb7b63755fd1562ddb0f26ee32a7a5c107d9209638cd8c150af3b1cc: Status 404 returned error can't find the container with id 46ca0084bb7b63755fd1562ddb0f26ee32a7a5c107d9209638cd8c150af3b1cc
Nov 03 19:36:31.986151 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:31.986117    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs" event={"ID":"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da","Type":"ContainerStarted","Data":"46ca0084bb7b63755fd1562ddb0f26ee32a7a5c107d9209638cd8c150af3b1cc"}
Nov 03 19:36:32.766908 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:36:32.766872    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:36:32.992272 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:32.992232    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs" event={"ID":"e4d4d3cd-6773-4b3b-80c6-52f5f9af92da","Type":"ContainerStarted","Data":"569af42a2c833a77957cf8e4e138dda7d2d0ea44695feb36e764696227c26c00"}
Nov 03 19:36:33.013766 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:33.013714    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs" podStartSLOduration=2.388174041 podStartE2EDuration="3.013699524s" totalImagesPullingTime="625.525483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:36:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:36:33.012656208 +0000 UTC m=+593.920678166" watchObservedRunningTime="2026-11-03 19:36:33.013699524 +0000 UTC m=+593.921721462"
Nov 03 19:36:33.207949 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:33.207856    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:36:33.292415 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:33.292373    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:35.355380 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.355344    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"]
Nov 03 19:36:35.359934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.359910    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.365109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.365087    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Nov 03 19:36:35.371314 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.371288    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"]
Nov 03 19:36:35.459671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.459641    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.459813 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.459685    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e2957a8-5aed-4931-a478-eadcd6eaa562-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.459813 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.459704    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss94q\" (UniqueName: \"kubernetes.io/projected/9e2957a8-5aed-4931-a478-eadcd6eaa562-kube-api-access-ss94q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.459883 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.459832    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.459883 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.459860    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.459991 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.459974    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.560786 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.560757    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.560898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.560803    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.560898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.560841    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e2957a8-5aed-4931-a478-eadcd6eaa562-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.560898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.560859    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss94q\" (UniqueName: \"kubernetes.io/projected/9e2957a8-5aed-4931-a478-eadcd6eaa562-kube-api-access-ss94q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.561159 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.560897    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.561159 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.560921    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.561275 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.561233    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.561348 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.561322    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.561415 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.561394    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.563231 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.563211    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e2957a8-5aed-4931-a478-eadcd6eaa562-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.563397 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.563379    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e2957a8-5aed-4931-a478-eadcd6eaa562-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.575109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.575083    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss94q\" (UniqueName: \"kubernetes.io/projected/9e2957a8-5aed-4931-a478-eadcd6eaa562-kube-api-access-ss94q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf\" (UID: \"9e2957a8-5aed-4931-a478-eadcd6eaa562\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.681518 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.681449    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:35.953009 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:35.952984    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"]
Nov 03 19:36:35.955560 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:36:35.955534    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2957a8_5aed_4931_a478_eadcd6eaa562.slice/crio-24d6bb8e9f38a1ad513ce71ee271cb2f9f1896353f6bd08cfbc07d25161394ae WatchSource:0}: Error finding container 24d6bb8e9f38a1ad513ce71ee271cb2f9f1896353f6bd08cfbc07d25161394ae: Status 404 returned error can't find the container with id 24d6bb8e9f38a1ad513ce71ee271cb2f9f1896353f6bd08cfbc07d25161394ae
Nov 03 19:36:36.009916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:36.009879    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf" event={"ID":"9e2957a8-5aed-4931-a478-eadcd6eaa562","Type":"ContainerStarted","Data":"24d6bb8e9f38a1ad513ce71ee271cb2f9f1896353f6bd08cfbc07d25161394ae"}
Nov 03 19:36:36.382710 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:36.382681    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:36:37.016686 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.016648    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf" event={"ID":"9e2957a8-5aed-4931-a478-eadcd6eaa562","Type":"ContainerStarted","Data":"423cf8bd46314d1c87b04be0592207dab794e24175c8377c71a7f4602ad6e47a"}
Nov 03 19:36:37.038238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.038188    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf" podStartSLOduration=1.684725287 podStartE2EDuration="2.03817473s" totalImagesPullingTime="353.449443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:36:37.03638131 +0000 UTC m=+597.944403262" watchObservedRunningTime="2026-11-03 19:36:37.03817473 +0000 UTC m=+597.946196668"
Nov 03 19:36:37.681912 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.681877    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:37.727091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.727058    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"]
Nov 03 19:36:37.731401 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.731380    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.739412 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.739386    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Nov 03 19:36:37.741626 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.741603    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"]
Nov 03 19:36:37.883766 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.883722    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdjvh\" (UniqueName: \"kubernetes.io/projected/1af7e503-62a3-4481-9c70-b24c8313c154-kube-api-access-mdjvh\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.883766 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.883770    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1af7e503-62a3-4481-9c70-b24c8313c154-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.883985 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.883856    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.883985 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.883890    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.883985 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.883929    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.884133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.884030    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.899699 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.899670    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:37.918086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.918055    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:37.984815 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.984783    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.984994 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.984840    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.984994 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.984893    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdjvh\" (UniqueName: \"kubernetes.io/projected/1af7e503-62a3-4481-9c70-b24c8313c154-kube-api-access-mdjvh\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.984994 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.984930    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1af7e503-62a3-4481-9c70-b24c8313c154-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.984994 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.984955    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.984994 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.984975    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.985372 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.985346    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.985444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.985388    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.985500 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.985449    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.987180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.987161    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1af7e503-62a3-4481-9c70-b24c8313c154-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.987651 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.987633    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1af7e503-62a3-4481-9c70-b24c8313c154-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:37.998029 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:37.998000    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdjvh\" (UniqueName: \"kubernetes.io/projected/1af7e503-62a3-4481-9c70-b24c8313c154-kube-api-access-mdjvh\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8\" (UID: \"1af7e503-62a3-4481-9c70-b24c8313c154\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:38.032726 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:38.032701    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n"
Nov 03 19:36:38.045194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:38.045171    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:38.329896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:38.329872    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"]
Nov 03 19:36:38.331508 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:36:38.331476    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1af7e503_62a3_4481_9c70_b24c8313c154.slice/crio-d391a2380f9306f37e26c753a0f8f0c9c48e108120895a753031ffacdd6a0769 WatchSource:0}: Error finding container d391a2380f9306f37e26c753a0f8f0c9c48e108120895a753031ffacdd6a0769: Status 404 returned error can't find the container with id d391a2380f9306f37e26c753a0f8f0c9c48e108120895a753031ffacdd6a0769
Nov 03 19:36:39.027941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:39.027901    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8" event={"ID":"1af7e503-62a3-4481-9c70-b24c8313c154","Type":"ContainerStarted","Data":"fef47e8388c9076f09546b857150213456bd1ea781aa2d4ff5373bed1959359d"}
Nov 03 19:36:39.028446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:39.027960    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8" event={"ID":"1af7e503-62a3-4481-9c70-b24c8313c154","Type":"ContainerStarted","Data":"d391a2380f9306f37e26c753a0f8f0c9c48e108120895a753031ffacdd6a0769"}
Nov 03 19:36:39.046246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:39.046218    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:39.050526 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:39.050479    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8" podStartSLOduration=1.725476587 podStartE2EDuration="2.050466823s" totalImagesPullingTime="324.990236ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:36:39.049734445 +0000 UTC m=+599.957756403" watchObservedRunningTime="2026-11-03 19:36:39.050466823 +0000 UTC m=+599.958488762"
Nov 03 19:36:39.266962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:39.266937    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:36:39.270157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:39.270130    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:36:39.980995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:39.980956    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:36:41.292631 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:41.292599    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:41.304923 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:41.304900    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:42.053310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:42.053283    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs"
Nov 03 19:36:44.569454 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:44.569422    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:36:45.681968 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:45.681933    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:45.694459 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:45.694438    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:46.073129 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:46.073105    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf"
Nov 03 19:36:48.045710 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:48.045679    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:48.058443 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:48.058418    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:48.082425 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:48.082404    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8"
Nov 03 19:36:54.255317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.255280    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-74f5787476-spqv7"]
Nov 03 19:36:54.258960 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.258942    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:36:54.269684 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.267005    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 03 19:36:54.275785 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.271414    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Nov 03 19:36:54.275785 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.271505    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-75sq7\""
Nov 03 19:36:54.275785 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.274060    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-74f5787476-spqv7"]
Nov 03 19:36:54.332213 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.332182    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4jq7\" (UniqueName: \"kubernetes.io/projected/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-kube-api-access-x4jq7\") pod \"ai-gateway-controller-74f5787476-spqv7\" (UID: \"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8\") " pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:36:54.332353 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.332254    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-metrics-certs\") pod \"ai-gateway-controller-74f5787476-spqv7\" (UID: \"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8\") " pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:36:54.433304 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.433273    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4jq7\" (UniqueName: \"kubernetes.io/projected/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-kube-api-access-x4jq7\") pod \"ai-gateway-controller-74f5787476-spqv7\" (UID: \"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8\") " pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:36:54.433433 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.433359    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-metrics-certs\") pod \"ai-gateway-controller-74f5787476-spqv7\" (UID: \"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8\") " pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:36:54.435769 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.435741    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-metrics-certs\") pod \"ai-gateway-controller-74f5787476-spqv7\" (UID: \"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8\") " pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:36:54.446013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.445991    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4jq7\" (UniqueName: \"kubernetes.io/projected/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-kube-api-access-x4jq7\") pod \"ai-gateway-controller-74f5787476-spqv7\" (UID: \"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8\") " pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:36:54.577981 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.577906    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:36:54.852270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:54.852246    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-74f5787476-spqv7"]
Nov 03 19:36:54.854083 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:36:54.854049    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb987d37b_7ad3_4229_bd5a_d84f2f7ca5a8.slice/crio-ce9b1d34776b55bca956ad06f952e355d21f412121267d3caf1cfb23cebc4c70 WatchSource:0}: Error finding container ce9b1d34776b55bca956ad06f952e355d21f412121267d3caf1cfb23cebc4c70: Status 404 returned error can't find the container with id ce9b1d34776b55bca956ad06f952e355d21f412121267d3caf1cfb23cebc4c70
Nov 03 19:36:55.103997 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.103918    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7" event={"ID":"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8","Type":"ContainerStarted","Data":"ce9b1d34776b55bca956ad06f952e355d21f412121267d3caf1cfb23cebc4c70"}
Nov 03 19:36:55.478300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.478264    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"]
Nov 03 19:36:55.483129 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.483112    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.488481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.488460    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 03 19:36:55.488481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.488459    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ndffd\""
Nov 03 19:36:55.500891 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.500870    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"]
Nov 03 19:36:55.543914 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.543889    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.544014 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.543918    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.544014 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.543942    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.544173 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.544152    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.544228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.544209    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht7sw\" (UniqueName: \"kubernetes.io/projected/b25cf9ad-db43-42cb-8fc8-883c249e8796-kube-api-access-ht7sw\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.544267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.544238    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.544317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.544268    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25cf9ad-db43-42cb-8fc8-883c249e8796-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.544317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.544287    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.544421 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.544396    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.645560 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.645537    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.645664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.645569    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.645664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.645597    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.645664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.645641    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.645664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.645659    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht7sw\" (UniqueName: \"kubernetes.io/projected/b25cf9ad-db43-42cb-8fc8-883c249e8796-kube-api-access-ht7sw\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.645862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.645679    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.645862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.645707    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25cf9ad-db43-42cb-8fc8-883c249e8796-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.645862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.645733    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.645862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.645781    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.646152 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.646102    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.646282 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.646159    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.646282 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.646187    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.646282 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.646266    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.646875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.646851    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25cf9ad-db43-42cb-8fc8-883c249e8796-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.648172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.648148    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.648397 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.648378    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.658719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.658695    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25cf9ad-db43-42cb-8fc8-883c249e8796-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.658837 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.658777    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht7sw\" (UniqueName: \"kubernetes.io/projected/b25cf9ad-db43-42cb-8fc8-883c249e8796-kube-api-access-ht7sw\") pod \"maas-default-gateway-openshift-default-66db998887-c684f\" (UID: \"b25cf9ad-db43-42cb-8fc8-883c249e8796\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:55.796740 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:55.796667    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:36:56.088493 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:56.088417    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"]
Nov 03 19:36:56.091089 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:36:56.091057    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25cf9ad_db43_42cb_8fc8_883c249e8796.slice/crio-85163763acd317e3dbe343ff8df98b596750023402389c2f0f01504559eedb50 WatchSource:0}: Error finding container 85163763acd317e3dbe343ff8df98b596750023402389c2f0f01504559eedb50: Status 404 returned error can't find the container with id 85163763acd317e3dbe343ff8df98b596750023402389c2f0f01504559eedb50
Nov 03 19:36:56.111790 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:56.111757    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" event={"ID":"b25cf9ad-db43-42cb-8fc8-883c249e8796","Type":"ContainerStarted","Data":"85163763acd317e3dbe343ff8df98b596750023402389c2f0f01504559eedb50"}
Nov 03 19:36:59.532222 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:59.532185    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:36:59.532575 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:59.532266    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:36:59.532575 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:36:59.532298    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:00.136585 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:00.136491    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" event={"ID":"b25cf9ad-db43-42cb-8fc8-883c249e8796","Type":"ContainerStarted","Data":"15dc5224928c9c3f0a8d6f9946fe5f5dd354555132795d1d7bc6b0c407b956b3"}
Nov 03 19:37:00.138137 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:00.138108    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7" event={"ID":"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8","Type":"ContainerStarted","Data":"d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b"}
Nov 03 19:37:00.138277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:00.138247    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:37:00.184483 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:00.184416    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podStartSLOduration=1.7468005469999999 podStartE2EDuration="5.184398944s" totalImagesPullingTime="3.437598397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:37:00.162290774 +0000 UTC m=+621.070312715" watchObservedRunningTime="2026-11-03 19:37:00.184398944 +0000 UTC m=+621.092420883"
Nov 03 19:37:00.185216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:00.185174    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7" podStartSLOduration=1.302444574 podStartE2EDuration="6.18516421s" totalImagesPullingTime="4.882719636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:36:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:37:00.184271232 +0000 UTC m=+621.092293171" watchObservedRunningTime="2026-11-03 19:37:00.18516421 +0000 UTC m=+621.093186149"
Nov 03 19:37:00.797201 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:00.797160    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:37:00.798658 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:00.798631    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Nov 03 19:37:00.798768 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:00.798689    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Nov 03 19:37:01.797112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:01.797070    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Nov 03 19:37:01.797295 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:01.797127    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Nov 03 19:37:02.788998 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:37:02.788964    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:37:02.797537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:02.797508    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Nov 03 19:37:02.797824 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:02.797554    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Nov 03 19:37:03.797481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:03.797446    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Nov 03 19:37:03.797658 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:03.797505    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Nov 03 19:37:04.797272 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:04.797232    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Nov 03 19:37:04.797455 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:04.797289    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Nov 03 19:37:05.798241 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:05.798199    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Nov 03 19:37:05.798648 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:05.798275    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Nov 03 19:37:06.797345 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:06.797303    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Nov 03 19:37:06.797514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:06.797371    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Nov 03 19:37:08.797521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:08.797468    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:37:08.798032 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:08.797531    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:37:09.839189 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:09.839150    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:37:09.839571 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:09.839266    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:37:10.013939 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.013902    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:37:10.014330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.014286    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:37:10.015152 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.015133    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f"
Nov 03 19:37:10.566971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.566937    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"]
Nov 03 19:37:10.572574 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.572548    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.583821 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.583793    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"]
Nov 03 19:37:10.698160 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.698121    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.698160 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.698171    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.698376 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.698261    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.698376 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.698301    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.698376 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.698329    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.698476 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.698375    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.698476 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.698415    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.698557 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.698514    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b1113b4-950d-4cf7-a856-1290aa4a1934-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.698557 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.698548    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bqh5\" (UniqueName: \"kubernetes.io/projected/9b1113b4-950d-4cf7-a856-1290aa4a1934-kube-api-access-7bqh5\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.798935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.798909    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.798944    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.798964    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.799002    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.799026    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799285 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.799083    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b1113b4-950d-4cf7-a856-1290aa4a1934-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799285 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.799121    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bqh5\" (UniqueName: \"kubernetes.io/projected/9b1113b4-950d-4cf7-a856-1290aa4a1934-kube-api-access-7bqh5\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799285 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.799159    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799285 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.799192    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.799380    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.799427    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799577 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.799483    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799691 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.799672    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.799836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.799817    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b1113b4-950d-4cf7-a856-1290aa4a1934-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.801380 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.801353    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.801568 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.801549    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.812294 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.812271    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1113b4-950d-4cf7-a856-1290aa4a1934-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.812485 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.812467    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bqh5\" (UniqueName: \"kubernetes.io/projected/9b1113b4-950d-4cf7-a856-1290aa4a1934-kube-api-access-7bqh5\") pod \"maas-default-gateway-openshift-default-66db998887-xtsbv\" (UID: \"9b1113b4-950d-4cf7-a856-1290aa4a1934\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:10.889473 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:10.889417    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:11.153082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.152994    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:37:11.169853 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.169831    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"]
Nov 03 19:37:11.171620 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:37:11.171594    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b1113b4_950d_4cf7_a856_1290aa4a1934.slice/crio-80c6c4b91ee85c58beeecf6a35d0dbdba786ab6b803e716986267e3fba866191 WatchSource:0}: Error finding container 80c6c4b91ee85c58beeecf6a35d0dbdba786ab6b803e716986267e3fba866191: Status 404 returned error can't find the container with id 80c6c4b91ee85c58beeecf6a35d0dbdba786ab6b803e716986267e3fba866191
Nov 03 19:37:11.173674 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.173646    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:11.173760 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.173703    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:11.173802 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.173757    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:11.215890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.215862    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" event={"ID":"9b1113b4-950d-4cf7-a856-1290aa4a1934","Type":"ContainerStarted","Data":"80c6c4b91ee85c58beeecf6a35d0dbdba786ab6b803e716986267e3fba866191"}
Nov 03 19:37:11.458162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.458073    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"]
Nov 03 19:37:11.461853 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.461831    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"
Nov 03 19:37:11.478120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.478098    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"]
Nov 03 19:37:11.505345 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.505316    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae94d23-3b48-4bf4-bc45-d4e248468c0f-metrics-certs\") pod \"ai-gateway-controller-f47bcb7b4-ftdw4\" (UID: \"5ae94d23-3b48-4bf4-bc45-d4e248468c0f\") " pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"
Nov 03 19:37:11.505518 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.505378    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j8ff\" (UniqueName: \"kubernetes.io/projected/5ae94d23-3b48-4bf4-bc45-d4e248468c0f-kube-api-access-8j8ff\") pod \"ai-gateway-controller-f47bcb7b4-ftdw4\" (UID: \"5ae94d23-3b48-4bf4-bc45-d4e248468c0f\") " pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"
Nov 03 19:37:11.606138 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.606007    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae94d23-3b48-4bf4-bc45-d4e248468c0f-metrics-certs\") pod \"ai-gateway-controller-f47bcb7b4-ftdw4\" (UID: \"5ae94d23-3b48-4bf4-bc45-d4e248468c0f\") " pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"
Nov 03 19:37:11.606484 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.606445    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j8ff\" (UniqueName: \"kubernetes.io/projected/5ae94d23-3b48-4bf4-bc45-d4e248468c0f-kube-api-access-8j8ff\") pod \"ai-gateway-controller-f47bcb7b4-ftdw4\" (UID: \"5ae94d23-3b48-4bf4-bc45-d4e248468c0f\") " pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"
Nov 03 19:37:11.609600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.609572    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ae94d23-3b48-4bf4-bc45-d4e248468c0f-metrics-certs\") pod \"ai-gateway-controller-f47bcb7b4-ftdw4\" (UID: \"5ae94d23-3b48-4bf4-bc45-d4e248468c0f\") " pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"
Nov 03 19:37:11.620108 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.620088    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j8ff\" (UniqueName: \"kubernetes.io/projected/5ae94d23-3b48-4bf4-bc45-d4e248468c0f-kube-api-access-8j8ff\") pod \"ai-gateway-controller-f47bcb7b4-ftdw4\" (UID: \"5ae94d23-3b48-4bf4-bc45-d4e248468c0f\") " pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"
Nov 03 19:37:11.775474 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:11.775435    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"
Nov 03 19:37:12.072723 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:12.072687    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"]
Nov 03 19:37:12.222076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:12.222027    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4" event={"ID":"5ae94d23-3b48-4bf4-bc45-d4e248468c0f","Type":"ContainerStarted","Data":"4d00badc4564f50aa874eaf0d4f99fb813297d5e479c367854dc5091d9189db4"}
Nov 03 19:37:12.223634 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:12.223607    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" event={"ID":"9b1113b4-950d-4cf7-a856-1290aa4a1934","Type":"ContainerStarted","Data":"10838bb0d834b6858581fa95eeeeb72a1aa24ebe1570996ac3fd25b0a92d21b2"}
Nov 03 19:37:12.266014 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:12.265950    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" podStartSLOduration=2.265930789 podStartE2EDuration="2.265930789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:37:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:37:12.265348928 +0000 UTC m=+633.173370868" watchObservedRunningTime="2026-11-03 19:37:12.265930789 +0000 UTC m=+633.173952730"
Nov 03 19:37:12.890228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:12.890150    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:12.891854 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:12.891823    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xtsbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 03 19:37:12.891989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:12.891868    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" podUID="9b1113b4-950d-4cf7-a856-1290aa4a1934" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 03 19:37:13.229706 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:13.229665    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4" event={"ID":"5ae94d23-3b48-4bf4-bc45-d4e248468c0f","Type":"ContainerStarted","Data":"7fab027287a206dac405ba9e6921137441bc8a22f30d090c27be4985e04777be"}
Nov 03 19:37:13.266119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:13.266061    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4" podStartSLOduration=1.728829537 podStartE2EDuration="2.266034265s" totalImagesPullingTime="537.204728ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:37:13.265412423 +0000 UTC m=+634.173434363" watchObservedRunningTime="2026-11-03 19:37:13.266034265 +0000 UTC m=+634.174056205"
Nov 03 19:37:13.890112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:13.890073    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xtsbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 03 19:37:13.890284 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:13.890132    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" podUID="9b1113b4-950d-4cf7-a856-1290aa4a1934" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 03 19:37:14.234376 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:14.234347    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"
Nov 03 19:37:14.890563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:14.890525    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xtsbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 03 19:37:14.890739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:14.890591    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" podUID="9b1113b4-950d-4cf7-a856-1290aa4a1934" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 03 19:37:15.890120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:15.890036    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xtsbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 03 19:37:15.890576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:15.890140    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" podUID="9b1113b4-950d-4cf7-a856-1290aa4a1934" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 03 19:37:16.890651 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:16.890617    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xtsbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 03 19:37:16.891067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:16.890670    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" podUID="9b1113b4-950d-4cf7-a856-1290aa4a1934" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 03 19:37:17.890419 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:17.890383    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xtsbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 03 19:37:17.890602 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:17.890439    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" podUID="9b1113b4-950d-4cf7-a856-1290aa4a1934" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 03 19:37:19.351660 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:19.351617    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:19.352103 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:19.351944    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:19.352870 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:19.352844    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv"
Nov 03 19:37:25.241077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.241025    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-f47bcb7b4-ftdw4"
Nov 03 19:37:25.286584 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.286552    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-74f5787476-spqv7"]
Nov 03 19:37:25.286771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.286743    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7" podUID="b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8" containerName="manager" containerID="cri-o://d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b" gracePeriod=30
Nov 03 19:37:25.530484 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.530463    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:37:25.582970 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.582938    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"]
Nov 03 19:37:25.583392 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.583378    5051 cpu_manager.go:414] "containerMap: removing container" podUID="b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8" containerName="manager"
Nov 03 19:37:25.583436 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.583394    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8" containerName="manager"
Nov 03 19:37:25.583497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.583487    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8" containerName="manager"
Nov 03 19:37:25.586589 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.586570    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.600594 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.600574    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"]
Nov 03 19:37:25.667109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.667063    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-metrics-certs\") pod \"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8\" (UID: \"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8\") "
Nov 03 19:37:25.667279 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.667131    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-kube-api-access-x4jq7\" (UniqueName: \"kubernetes.io/projected/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-kube-api-access-x4jq7\") pod \"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8\" (UID: \"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8\") "
Nov 03 19:37:25.669201 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.669178    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-metrics-certs" pod "b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8" (UID: "b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:37:25.669349 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.669327    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-kube-api-access-x4jq7" pod "b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8" (UID: "b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8"). InnerVolumeSpecName "kube-api-access-x4jq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:37:25.768325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.768262    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.768325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.768302    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwkqz\" (UniqueName: \"kubernetes.io/projected/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-kube-api-access-dwkqz\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.768488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.768334    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.768488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.768413    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.768488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.768446    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.768642 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.768594    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.768688 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.768639    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.768735 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.768713    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.768774 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.768757    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.768823 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.768813    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:37:25.768870 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.768826    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4jq7\" (UniqueName: \"kubernetes.io/projected/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8-kube-api-access-x4jq7\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:37:25.870171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870138    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870239    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870271    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870307    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870493 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870351    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870493 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870380    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870493 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870409    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwkqz\" (UniqueName: \"kubernetes.io/projected/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-kube-api-access-dwkqz\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870493 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870451    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870493 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870481    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870744 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870546    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870744 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870619    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870848 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870783    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.870848 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.870815    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.871084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.871062    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.872599 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.872578    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.872879 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.872861    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.884774 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.884752    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.884774 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.884762    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwkqz\" (UniqueName: \"kubernetes.io/projected/159b05c6-8444-4ed2-95d7-7d07dd9d8b34-kube-api-access-dwkqz\") pod \"maas-default-gateway-openshift-default-66db998887-qtcsx\" (UID: \"159b05c6-8444-4ed2-95d7-7d07dd9d8b34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:25.899751 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:25.899736    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:26.178667 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.178639    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"]
Nov 03 19:37:26.180349 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:37:26.180323    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod159b05c6_8444_4ed2_95d7_7d07dd9d8b34.slice/crio-ac1a492e12c73c3f99fb7cd1080d4b0dac1d0416dd5c5a54d6a5c44821873892 WatchSource:0}: Error finding container ac1a492e12c73c3f99fb7cd1080d4b0dac1d0416dd5c5a54d6a5c44821873892: Status 404 returned error can't find the container with id ac1a492e12c73c3f99fb7cd1080d4b0dac1d0416dd5c5a54d6a5c44821873892
Nov 03 19:37:26.182479 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.182445    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:26.182602 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.182538    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:26.182602 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.182582    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:26.319205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.319175    5051 generic.go:356] "Generic (PLEG): container finished" podID="b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8" containerID="d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b" exitCode=0
Nov 03 19:37:26.319597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.319221    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7"
Nov 03 19:37:26.319597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.319258    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7" event={"ID":"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8","Type":"ContainerDied","Data":"d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b"}
Nov 03 19:37:26.319597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.319325    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-74f5787476-spqv7" event={"ID":"b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8","Type":"ContainerDied","Data":"ce9b1d34776b55bca956ad06f952e355d21f412121267d3caf1cfb23cebc4c70"}
Nov 03 19:37:26.319597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.319355    5051 scope.go:122] "RemoveContainer" containerID="d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b"
Nov 03 19:37:26.321868 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.321843    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx" event={"ID":"159b05c6-8444-4ed2-95d7-7d07dd9d8b34","Type":"ContainerStarted","Data":"1793a53c24a9807cb23fa106bf7d3c883dec92dd174c3e30769cd6b955053716"}
Nov 03 19:37:26.321993 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.321891    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx" event={"ID":"159b05c6-8444-4ed2-95d7-7d07dd9d8b34","Type":"ContainerStarted","Data":"ac1a492e12c73c3f99fb7cd1080d4b0dac1d0416dd5c5a54d6a5c44821873892"}
Nov 03 19:37:26.330402 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.330380    5051 scope.go:122] "RemoveContainer" containerID="d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b"
Nov 03 19:37:26.330684 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:37:26.330666    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b\": container with ID starting with d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b not found: ID does not exist" containerID="d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b"
Nov 03 19:37:26.330743 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.330694    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b"} err="failed to get container status \"d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b\": rpc error: code = NotFound desc = could not find container \"d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b\": container with ID starting with d96de4cd1ee54d91ee504e26dc0ad477b25d4c4fa47664bb34798d44b6132c1b not found: ID does not exist"
Nov 03 19:37:26.351206 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.351152    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx" podStartSLOduration=1.351136494 podStartE2EDuration="1.351136494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:37:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:37:26.346602673 +0000 UTC m=+647.254624612" watchObservedRunningTime="2026-11-03 19:37:26.351136494 +0000 UTC m=+647.259158434"
Nov 03 19:37:26.364470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.364448    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-74f5787476-spqv7"]
Nov 03 19:37:26.372763 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.372742    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-74f5787476-spqv7"]
Nov 03 19:37:26.900437 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:26.900396    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:27.227988 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:27.227950    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8" path="/var/lib/kubelet/pods/b987d37b-7ad3-4229-bd5a-d84f2f7ca5a8/volumes"
Nov 03 19:37:27.901705 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:27.901662    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qtcsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 03 19:37:27.902169 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:27.901721    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx" podUID="159b05c6-8444-4ed2-95d7-7d07dd9d8b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 03 19:37:28.900774 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:28.900737    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qtcsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 03 19:37:28.900958 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:28.900796    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx" podUID="159b05c6-8444-4ed2-95d7-7d07dd9d8b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 03 19:37:29.901123 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:29.901089    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qtcsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 03 19:37:29.901502 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:29.901140    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx" podUID="159b05c6-8444-4ed2-95d7-7d07dd9d8b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 03 19:37:30.900293 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:30.900257    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qtcsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 03 19:37:30.900454 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:30.900310    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx" podUID="159b05c6-8444-4ed2-95d7-7d07dd9d8b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 03 19:37:31.900560 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:31.900523    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qtcsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 03 19:37:31.901013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:31.900578    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx" podUID="159b05c6-8444-4ed2-95d7-7d07dd9d8b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 03 19:37:32.766657 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:37:32.766618    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:37:32.900333 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:32.900298    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qtcsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 03 19:37:32.900493 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:32.900352    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx" podUID="159b05c6-8444-4ed2-95d7-7d07dd9d8b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 03 19:37:34.356383 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:34.356352    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:34.356855 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:34.356714    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:34.357446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:34.357424    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx"
Nov 03 19:37:54.154816 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.154731    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"]
Nov 03 19:37:54.159234 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.159207    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.168117 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.168092    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-036184-openshift-default-dockercfg-xkkwv\""
Nov 03 19:37:54.174003 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.173978    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"]
Nov 03 19:37:54.228076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.228025    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.228199 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.228091    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.228199 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.228135    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.228273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.228250    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.228307 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.228276    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6nbh\" (UniqueName: \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-kube-api-access-d6nbh\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.228340 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.228310    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.228387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.228346    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.228430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.228390    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.228497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.228484    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.290054 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.290020    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"]
Nov 03 19:37:54.293710 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.293693    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.324306 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.324275    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-05e217-openshift-default-dockercfg-r9knx\""
Nov 03 19:37:54.329735 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.329702    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.329845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.329755    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6nbh\" (UniqueName: \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-kube-api-access-d6nbh\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.329845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.329801    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.329845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.329823    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.330018 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.329855    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.330018 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.329932    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.330018 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.329983    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.330018 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.330012    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.330316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.330037    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.330371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.330332    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.330424 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.330367    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.330782 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.330740    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.331160 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.331125    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.331282 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.331218    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.333405 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.333374    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.334225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.333857    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.334777 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.334706    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"]
Nov 03 19:37:54.346961 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.346936    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6nbh\" (UniqueName: \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-kube-api-access-d6nbh\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.355324 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.355298    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.430680 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.430596    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.430680 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.430637    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd2cx\" (UniqueName: \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-kube-api-access-dd2cx\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.430680 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.430675    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.430935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.430757    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.430935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.430804    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.430935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.430829    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.430935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.430875    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.431104 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.430983    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.431104 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.431027    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.473344 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.473306    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:54.532190 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.532149    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.532365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.532195    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd2cx\" (UniqueName: \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-kube-api-access-dd2cx\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.532365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.532252    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.532365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.532297    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.532365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.532348    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.532617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.532379    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.532617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.532405    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.532617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.532462    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.532617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.532517    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.535583 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.533438    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.535583 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.533658    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.535583 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.534196    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.535583 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.534663    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.535583 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.535451    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.537130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.537104    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.538792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.538768    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.583182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.583152    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.584798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.584770    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd2cx\" (UniqueName: \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-kube-api-access-dd2cx\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.606538 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.606513    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:54.757073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.757019    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"]
Nov 03 19:37:54.762616 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.762587    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.768747 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.768692    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"]
Nov 03 19:37:54.777384 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.776856    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"]
Nov 03 19:37:54.779996 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.779959    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:54.780128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.780056    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:54.780128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.780106    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:54.781286 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.780921    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-ef8f84-openshift-default-dockercfg-77kt7\""
Nov 03 19:37:54.834925 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.834902    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.835021 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.834931    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.835088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.835029    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.835147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.835126    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.835195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.835156    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.835195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.835174    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz8qv\" (UniqueName: \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-kube-api-access-qz8qv\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.835317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.835300    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.835375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.835346    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.835422 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.835381    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.854907 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.854879    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"]
Nov 03 19:37:54.859993 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.859968    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:54.877487 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.874421    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"]
Nov 03 19:37:54.881241 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.881202    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-c5f6cb-openshift-default-dockercfg-p6txp\""
Nov 03 19:37:54.899148 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.899001    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"]
Nov 03 19:37:54.907105 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.907036    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:54.907205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.907148    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:54.907205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.907197    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936562    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936615    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936642    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936669    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936708    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936736    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936774    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936806    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936839    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936879    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936920    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.936992    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.937018    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.937101    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.937474    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.937492    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.937534    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.937562    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz8qv\" (UniqueName: \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-kube-api-access-qz8qv\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.937737    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.937918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.937808    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.938950 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.937970    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.955381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.940322    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl4kx\" (UniqueName: \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-kube-api-access-dl4kx\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:54.955381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.942565    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.955381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.943961    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.955381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.945658    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.955381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.952919    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:54.966176 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:54.965695    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz8qv\" (UniqueName: \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-kube-api-access-qz8qv\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:55.041781 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.041688    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.041781 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.041752    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.041789    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.041839    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.041913    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.041974    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042295 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.042018    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl4kx\" (UniqueName: \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-kube-api-access-dl4kx\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042295 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.042119    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042295 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.042168    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042295 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.042167    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042499 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.042424    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042567 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.042532    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042715 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.042692    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.042979 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.042956    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.044854 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.044822    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.046137 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.046113    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.058195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.058166    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl4kx\" (UniqueName: \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-kube-api-access-dl4kx\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.058808 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.058788    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.086327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.086298    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:55.183271 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.179456    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:55.376636 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.376604    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"]
Nov 03 19:37:55.377591 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:37:55.377561    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ae4144_2795_4fee_b8c7_9a5859c55d35.slice/crio-bf394c1cb253bb83af959e898f939c09978ed52b1c3233ba538a186932c3b26e WatchSource:0}: Error finding container bf394c1cb253bb83af959e898f939c09978ed52b1c3233ba538a186932c3b26e: Status 404 returned error can't find the container with id bf394c1cb253bb83af959e898f939c09978ed52b1c3233ba538a186932c3b26e
Nov 03 19:37:55.379981 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.379936    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:55.380085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.380031    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:55.380131 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.380094    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:55.491427 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.491402    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"]
Nov 03 19:37:55.494508 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:37:55.494482    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd2e2038_3d81_4246_8ef8_c29bfb7e8942.slice/crio-f1476fca16202e4027b60b1bcaf855e713a0cc9ab4ce4c4d2c3a398c6f1b1bf5 WatchSource:0}: Error finding container f1476fca16202e4027b60b1bcaf855e713a0cc9ab4ce4c4d2c3a398c6f1b1bf5: Status 404 returned error can't find the container with id f1476fca16202e4027b60b1bcaf855e713a0cc9ab4ce4c4d2c3a398c6f1b1bf5
Nov 03 19:37:55.496190 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.496166    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" event={"ID":"e2ae4144-2795-4fee-b8c7-9a5859c55d35","Type":"ContainerStarted","Data":"cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927"}
Nov 03 19:37:55.496516 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.496214    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" event={"ID":"e2ae4144-2795-4fee-b8c7-9a5859c55d35","Type":"ContainerStarted","Data":"bf394c1cb253bb83af959e898f939c09978ed52b1c3233ba538a186932c3b26e"}
Nov 03 19:37:55.497313 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.497277    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:55.497435 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.497358    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:55.497435 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.497404    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:37:55.497716 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.497694    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" event={"ID":"332ebb89-3bbd-47f3-a47e-33138a5bc16e","Type":"ContainerStarted","Data":"a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364"}
Nov 03 19:37:55.497797 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.497750    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" event={"ID":"332ebb89-3bbd-47f3-a47e-33138a5bc16e","Type":"ContainerStarted","Data":"2cb5466fef359eaf1e9caa2b7d2b2b87c3d0841af2c2163b1f8b508d4e7d311b"}
Nov 03 19:37:55.499852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.499820    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" event={"ID":"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9","Type":"ContainerStarted","Data":"4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf"}
Nov 03 19:37:55.499954 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.499870    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" event={"ID":"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9","Type":"ContainerStarted","Data":"bf767bbcfe862a118d649d037de66b193b7120ea04f50f02d33035e5d00206c3"}
Nov 03 19:37:55.554009 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.553964    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" podStartSLOduration=1.553947886 podStartE2EDuration="1.553947886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:37:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:37:55.552873236 +0000 UTC m=+676.460895196" watchObservedRunningTime="2026-11-03 19:37:55.553947886 +0000 UTC m=+676.461969804"
Nov 03 19:37:55.555135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.554986    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" podStartSLOduration=1.554980145 podStartE2EDuration="1.554980145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:37:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:37:55.529993004 +0000 UTC m=+676.438014955" watchObservedRunningTime="2026-11-03 19:37:55.554980145 +0000 UTC m=+676.463002063"
Nov 03 19:37:55.578172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.576566    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" podStartSLOduration=1.576551539 podStartE2EDuration="1.576551539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:37:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:37:55.575559726 +0000 UTC m=+676.483581680" watchObservedRunningTime="2026-11-03 19:37:55.576551539 +0000 UTC m=+676.484573480"
Nov 03 19:37:55.607118 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.607077    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:55.612000 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:55.611976    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:56.086571 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.086538    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:56.092106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.092082    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:56.473475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.473446    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:56.481051 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.481013    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:56.506676 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.506640    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" event={"ID":"bd2e2038-3d81-4246-8ef8-c29bfb7e8942","Type":"ContainerStarted","Data":"ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237"}
Nov 03 19:37:56.506832 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.506702    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" event={"ID":"bd2e2038-3d81-4246-8ef8-c29bfb7e8942","Type":"ContainerStarted","Data":"f1476fca16202e4027b60b1bcaf855e713a0cc9ab4ce4c4d2c3a398c6f1b1bf5"}
Nov 03 19:37:56.507007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.506986    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:56.507535 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.507513    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:56.507634 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.507562    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:56.508035 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.508018    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:37:56.508134 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.508096    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:37:56.508596 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.508576    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:37:56.532292 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:56.532250    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" podStartSLOduration=2.532237551 podStartE2EDuration="2.532237551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:37:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:37:56.530558715 +0000 UTC m=+677.438580677" watchObservedRunningTime="2026-11-03 19:37:56.532237551 +0000 UTC m=+677.440259533"
Nov 03 19:37:57.180941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:57.180908    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:57.186500 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:57.186474    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:57.511990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:57.511960    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:37:57.512986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:37:57.512967    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:38:00.337758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.337722    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"]
Nov 03 19:38:00.342747 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.342730    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:38:00.353499 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.353476    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-036184\""
Nov 03 19:38:00.354788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.354732    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"]
Nov 03 19:38:00.355622 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.355598    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-036184-dockercfg-tnp56\""
Nov 03 19:38:00.505766 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.505733    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e39677-39a0-4a57-9672-dbf5d49e3168-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7\" (UID: \"c0e39677-39a0-4a57-9672-dbf5d49e3168\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:38:00.505938 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.505863    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qwlx\" (UniqueName: \"kubernetes.io/projected/c0e39677-39a0-4a57-9672-dbf5d49e3168-kube-api-access-8qwlx\") pod \"payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7\" (UID: \"c0e39677-39a0-4a57-9672-dbf5d49e3168\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:38:00.606618 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.606544    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qwlx\" (UniqueName: \"kubernetes.io/projected/c0e39677-39a0-4a57-9672-dbf5d49e3168-kube-api-access-8qwlx\") pod \"payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7\" (UID: \"c0e39677-39a0-4a57-9672-dbf5d49e3168\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:38:00.606764 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.606665    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e39677-39a0-4a57-9672-dbf5d49e3168-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7\" (UID: \"c0e39677-39a0-4a57-9672-dbf5d49e3168\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:38:00.607414 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.607394    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e39677-39a0-4a57-9672-dbf5d49e3168-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7\" (UID: \"c0e39677-39a0-4a57-9672-dbf5d49e3168\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:38:00.619474 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.619446    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qwlx\" (UniqueName: \"kubernetes.io/projected/c0e39677-39a0-4a57-9672-dbf5d49e3168-kube-api-access-8qwlx\") pod \"payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7\" (UID: \"c0e39677-39a0-4a57-9672-dbf5d49e3168\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:38:00.657020 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.656991    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:38:00.857977 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.857898    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"]
Nov 03 19:38:00.862834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.862817    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:38:00.872567 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.872543    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-ef8f84\""
Nov 03 19:38:00.874102 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.874080    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-ef8f84-dockercfg-t497w\""
Nov 03 19:38:00.876572 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.876549    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"]
Nov 03 19:38:00.900752 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.900726    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"]
Nov 03 19:38:00.904291 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.904276    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:38:00.915893 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.915828    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"]
Nov 03 19:38:00.933791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:00.933768    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"]
Nov 03 19:38:00.935818 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:38:00.935795    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0e39677_39a0_4a57_9672_dbf5d49e3168.slice/crio-eeb1bc0b06de721d53d64567fcc490d48dbbb519f32f168f77866ea281ccb8b3 WatchSource:0}: Error finding container eeb1bc0b06de721d53d64567fcc490d48dbbb519f32f168f77866ea281ccb8b3: Status 404 returned error can't find the container with id eeb1bc0b06de721d53d64567fcc490d48dbbb519f32f168f77866ea281ccb8b3
Nov 03 19:38:01.015281 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.015243    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4\" (UID: \"7ce60e5f-3d38-4e72-b857-bbdb675db1f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:38:01.015470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.015290    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htlpm\" (UniqueName: \"kubernetes.io/projected/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-kube-api-access-htlpm\") pod \"payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4\" (UID: \"7ce60e5f-3d38-4e72-b857-bbdb675db1f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:38:01.015470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.015356    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6c6d13b-5904-4d07-8534-4b724313bb79-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp\" (UID: \"b6c6d13b-5904-4d07-8534-4b724313bb79\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:38:01.015470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.015437    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kst6f\" (UniqueName: \"kubernetes.io/projected/b6c6d13b-5904-4d07-8534-4b724313bb79-kube-api-access-kst6f\") pod \"payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp\" (UID: \"b6c6d13b-5904-4d07-8534-4b724313bb79\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:38:01.116765 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.116684    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kst6f\" (UniqueName: \"kubernetes.io/projected/b6c6d13b-5904-4d07-8534-4b724313bb79-kube-api-access-kst6f\") pod \"payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp\" (UID: \"b6c6d13b-5904-4d07-8534-4b724313bb79\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:38:01.116935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.116802    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4\" (UID: \"7ce60e5f-3d38-4e72-b857-bbdb675db1f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:38:01.116935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.116829    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htlpm\" (UniqueName: \"kubernetes.io/projected/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-kube-api-access-htlpm\") pod \"payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4\" (UID: \"7ce60e5f-3d38-4e72-b857-bbdb675db1f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:38:01.116935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.116856    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6c6d13b-5904-4d07-8534-4b724313bb79-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp\" (UID: \"b6c6d13b-5904-4d07-8534-4b724313bb79\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:38:01.117574 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.117551    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6c6d13b-5904-4d07-8534-4b724313bb79-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp\" (UID: \"b6c6d13b-5904-4d07-8534-4b724313bb79\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:38:01.117742 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.117719    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4\" (UID: \"7ce60e5f-3d38-4e72-b857-bbdb675db1f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:38:01.129431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.129387    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kst6f\" (UniqueName: \"kubernetes.io/projected/b6c6d13b-5904-4d07-8534-4b724313bb79-kube-api-access-kst6f\") pod \"payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp\" (UID: \"b6c6d13b-5904-4d07-8534-4b724313bb79\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:38:01.130991 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.130969    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htlpm\" (UniqueName: \"kubernetes.io/projected/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-kube-api-access-htlpm\") pod \"payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4\" (UID: \"7ce60e5f-3d38-4e72-b857-bbdb675db1f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:38:01.176959 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.176937    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:38:01.221503 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.221276    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:38:01.484213 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.484091    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"]
Nov 03 19:38:01.539196 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.539154    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4" event={"ID":"7ce60e5f-3d38-4e72-b857-bbdb675db1f5","Type":"ContainerStarted","Data":"419ab7c5ce32c3d9f98bb4e7fc3901aba47e43d7b334383ce92ff2b1163764d4"}
Nov 03 19:38:01.547141 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.547116    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"]
Nov 03 19:38:01.547590 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.547551    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7" event={"ID":"c0e39677-39a0-4a57-9672-dbf5d49e3168","Type":"ContainerStarted","Data":"eeb1bc0b06de721d53d64567fcc490d48dbbb519f32f168f77866ea281ccb8b3"}
Nov 03 19:38:01.550411 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:38:01.550385    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6c6d13b_5904_4d07_8534_4b724313bb79.slice/crio-ca711630074bd39f972b19b000df94ec3c31c4669a79d47a71988ee2570bf29d WatchSource:0}: Error finding container ca711630074bd39f972b19b000df94ec3c31c4669a79d47a71988ee2570bf29d: Status 404 returned error can't find the container with id ca711630074bd39f972b19b000df94ec3c31c4669a79d47a71988ee2570bf29d
Nov 03 19:38:01.674268 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.674228    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"]
Nov 03 19:38:01.678872 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.678845    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:38:01.689916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.689892    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"]
Nov 03 19:38:01.690488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.690470    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-2c3fc4-dockercfg-msv5z\""
Nov 03 19:38:01.756221 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.756145    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-2c3fc4\""
Nov 03 19:38:01.825367 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.825328    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvdpc\" (UniqueName: \"kubernetes.io/projected/82b74e21-57a7-489f-abe4-9814cf683aab-kube-api-access-kvdpc\") pod \"payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m\" (UID: \"82b74e21-57a7-489f-abe4-9814cf683aab\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:38:01.825554 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.825489    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b74e21-57a7-489f-abe4-9814cf683aab-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m\" (UID: \"82b74e21-57a7-489f-abe4-9814cf683aab\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:38:01.926715 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.926677    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvdpc\" (UniqueName: \"kubernetes.io/projected/82b74e21-57a7-489f-abe4-9814cf683aab-kube-api-access-kvdpc\") pod \"payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m\" (UID: \"82b74e21-57a7-489f-abe4-9814cf683aab\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:38:01.926924 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.926779    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b74e21-57a7-489f-abe4-9814cf683aab-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m\" (UID: \"82b74e21-57a7-489f-abe4-9814cf683aab\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:38:01.927960 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.927916    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b74e21-57a7-489f-abe4-9814cf683aab-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m\" (UID: \"82b74e21-57a7-489f-abe4-9814cf683aab\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:38:01.940147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.940117    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvdpc\" (UniqueName: \"kubernetes.io/projected/82b74e21-57a7-489f-abe4-9814cf683aab-kube-api-access-kvdpc\") pod \"payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m\" (UID: \"82b74e21-57a7-489f-abe4-9814cf683aab\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:38:01.989105 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.989074    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:38:01.998674 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:01.998651    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:38:02.298683 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.298570    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"]
Nov 03 19:38:02.310085 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:38:02.310033    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b74e21_57a7_489f_abe4_9814cf683aab.slice/crio-52b459e97f796046fd0d6dcc52eccaf47f36eecf85bf08a8a355bfbad6d6477b WatchSource:0}: Error finding container 52b459e97f796046fd0d6dcc52eccaf47f36eecf85bf08a8a355bfbad6d6477b: Status 404 returned error can't find the container with id 52b459e97f796046fd0d6dcc52eccaf47f36eecf85bf08a8a355bfbad6d6477b
Nov 03 19:38:02.540080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.533276    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"]
Nov 03 19:38:02.540080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.538911    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.543462 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.543437    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-2c3fc4-serving-cert\""
Nov 03 19:38:02.562254 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.559530    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"]
Nov 03 19:38:02.573672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.573640    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp" event={"ID":"b6c6d13b-5904-4d07-8534-4b724313bb79","Type":"ContainerStarted","Data":"ca711630074bd39f972b19b000df94ec3c31c4669a79d47a71988ee2570bf29d"}
Nov 03 19:38:02.578845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.578813    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m" event={"ID":"82b74e21-57a7-489f-abe4-9814cf683aab","Type":"ContainerStarted","Data":"52b459e97f796046fd0d6dcc52eccaf47f36eecf85bf08a8a355bfbad6d6477b"}
Nov 03 19:38:02.636417 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.636386    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/610d96fe-f68f-4876-b85a-de3017180cf5-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.636670 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.636650    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-metrics-certs\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.636888 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.636861    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g24vl\" (UniqueName: \"kubernetes.io/projected/610d96fe-f68f-4876-b85a-de3017180cf5-kube-api-access-g24vl\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.637134 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.637114    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-maas-api-tls\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.738541 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.738471    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g24vl\" (UniqueName: \"kubernetes.io/projected/610d96fe-f68f-4876-b85a-de3017180cf5-kube-api-access-g24vl\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.739944 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.739208    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-maas-api-tls\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.739944 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.739250    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/610d96fe-f68f-4876-b85a-de3017180cf5-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.739944 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.739316    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-metrics-certs\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.739944 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:38:02.739928    5051 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-serving-cert: secret "maas-api-e2e-worker-w1-2c3fc4-serving-cert" not found
Nov 03 19:38:02.740693 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:38:02.739996    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-maas-api-tls podName:610d96fe-f68f-4876-b85a-de3017180cf5 nodeName:}" failed. No retries permitted until 2026-11-03 19:38:03.239973827 +0000 UTC m=+684.147995751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-maas-api-tls") pod "maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs" (UID: "610d96fe-f68f-4876-b85a-de3017180cf5") : secret "maas-api-e2e-worker-w1-2c3fc4-serving-cert" not found
Nov 03 19:38:02.740909 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.740881    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/610d96fe-f68f-4876-b85a-de3017180cf5-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.745190 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.745144    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-metrics-certs\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.752327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.752075    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g24vl\" (UniqueName: \"kubernetes.io/projected/610d96fe-f68f-4876-b85a-de3017180cf5-kube-api-access-g24vl\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:02.782357 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:38:02.782317    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:38:02.897471 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.895347    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"]
Nov 03 19:38:02.899988 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.899664    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:38:02.909701 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:02.909678    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"]
Nov 03 19:38:03.000149 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.000114    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-d8a4ea\""
Nov 03 19:38:03.053893 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.053562    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx\" (UID: \"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:38:03.053893 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.053661    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgq6r\" (UniqueName: \"kubernetes.io/projected/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-kube-api-access-jgq6r\") pod \"payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx\" (UID: \"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:38:03.155528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.155450    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx\" (UID: \"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:38:03.155682 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.155534    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgq6r\" (UniqueName: \"kubernetes.io/projected/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-kube-api-access-jgq6r\") pod \"payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx\" (UID: \"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:38:03.156361 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.156329    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx\" (UID: \"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:38:03.177993 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.177965    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgq6r\" (UniqueName: \"kubernetes.io/projected/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-kube-api-access-jgq6r\") pod \"payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx\" (UID: \"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:38:03.247157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.238361    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:38:03.257134 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.256500    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-maas-api-tls\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:03.273162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.273108    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-maas-api-tls\") pod \"maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:03.512124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.512090    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:03.574800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.574771    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"]
Nov 03 19:38:03.613686 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.613653    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx" event={"ID":"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac","Type":"ContainerStarted","Data":"a9074986ac38c19de429645144ac703bdff11e21ab06bd6a317eb918f67d3cfd"}
Nov 03 19:38:03.833631 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:03.833535    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"]
Nov 03 19:38:04.628280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:04.627619    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs" event={"ID":"610d96fe-f68f-4876-b85a-de3017180cf5","Type":"ContainerStarted","Data":"40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29"}
Nov 03 19:38:04.628280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:04.627691    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs" event={"ID":"610d96fe-f68f-4876-b85a-de3017180cf5","Type":"ContainerStarted","Data":"ffb933237b38efb12747ac20cd09100135ef295dfe43ad3838e1f40a7b354008"}
Nov 03 19:38:04.629137 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:04.629111    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:04.670500 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:04.670418    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs" podStartSLOduration=2.196706591 podStartE2EDuration="2.670398093s" totalImagesPullingTime="473.691502ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:38:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:38:04.662446402 +0000 UTC m=+685.570468368" watchObservedRunningTime="2026-11-03 19:38:04.670398093 +0000 UTC m=+685.578420016"
Nov 03 19:38:05.479164 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.479108    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"]
Nov 03 19:38:05.489921 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.489890    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.493722 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.493628    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"]
Nov 03 19:38:05.499792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.499586    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-c5f6cb-serving-cert\""
Nov 03 19:38:05.524080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.521691    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245556da-83c7-4339-8d4b-9be9c738a8e0-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.524080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.521755    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-metrics-certs\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.524080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.521844    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks77k\" (UniqueName: \"kubernetes.io/projected/245556da-83c7-4339-8d4b-9be9c738a8e0-kube-api-access-ks77k\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.524080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.521968    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-maas-api-tls\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.624578 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.623644    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-maas-api-tls\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.624578 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.623776    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245556da-83c7-4339-8d4b-9be9c738a8e0-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.624578 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.623804    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-metrics-certs\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.624578 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.623833    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks77k\" (UniqueName: \"kubernetes.io/projected/245556da-83c7-4339-8d4b-9be9c738a8e0-kube-api-access-ks77k\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.627357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.627316    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245556da-83c7-4339-8d4b-9be9c738a8e0-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.628148 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.628129    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-metrics-certs\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.628969 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.628868    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-maas-api-tls\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.692524 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.692498    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks77k\" (UniqueName: \"kubernetes.io/projected/245556da-83c7-4339-8d4b-9be9c738a8e0-kube-api-access-ks77k\") pod \"maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:05.812122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:05.811273    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:06.173783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.173755    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"]
Nov 03 19:38:06.175902 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.174943    5051 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 19:38:06.263140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.262996    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"]
Nov 03 19:38:06.298344 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.298315    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"]
Nov 03 19:38:06.298732 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.298712    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.306254 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.305780    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-05e217-serving-cert\""
Nov 03 19:38:06.356395 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.356248    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-metrics-certs\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.356580 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.356403    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87vc2\" (UniqueName: \"kubernetes.io/projected/35da5897-e86e-4951-8199-1f50f4d6b2e4-kube-api-access-87vc2\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.356580 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.356442    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-maas-api-tls\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.356699 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.356582    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35da5897-e86e-4951-8199-1f50f4d6b2e4-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.463111 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.462681    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87vc2\" (UniqueName: \"kubernetes.io/projected/35da5897-e86e-4951-8199-1f50f4d6b2e4-kube-api-access-87vc2\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.463111 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.462736    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-maas-api-tls\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.463111 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.462810    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35da5897-e86e-4951-8199-1f50f4d6b2e4-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.463111 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.462897    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-metrics-certs\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.490105 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.489291    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35da5897-e86e-4951-8199-1f50f4d6b2e4-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.506068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.506016    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-metrics-certs\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.508457 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.508431    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-maas-api-tls\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.511032 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.510931    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87vc2\" (UniqueName: \"kubernetes.io/projected/35da5897-e86e-4951-8199-1f50f4d6b2e4-kube-api-access-87vc2\") pod \"maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.700063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.697435    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:06.708108 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:06.707877    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz" event={"ID":"245556da-83c7-4339-8d4b-9be9c738a8e0","Type":"ContainerStarted","Data":"54466e3128264ea9dd8ffa57efa93f6a3d40302109c22dc5b243594a9194edef"}
Nov 03 19:38:07.059063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:07.058005    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"]
Nov 03 19:38:07.112092 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:38:07.112033    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35da5897_e86e_4951_8199_1f50f4d6b2e4.slice/crio-5a7eafb8deaf525c189efa3b91e8ccf4cf1cae4da01a9b8f6f9d2f38cc592610 WatchSource:0}: Error finding container 5a7eafb8deaf525c189efa3b91e8ccf4cf1cae4da01a9b8f6f9d2f38cc592610: Status 404 returned error can't find the container with id 5a7eafb8deaf525c189efa3b91e8ccf4cf1cae4da01a9b8f6f9d2f38cc592610
Nov 03 19:38:07.739074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:07.730957    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7" event={"ID":"c0e39677-39a0-4a57-9672-dbf5d49e3168","Type":"ContainerStarted","Data":"14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0"}
Nov 03 19:38:07.739074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:07.731843    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:38:07.739074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:07.733842    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz" event={"ID":"245556da-83c7-4339-8d4b-9be9c738a8e0","Type":"ContainerStarted","Data":"f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c"}
Nov 03 19:38:07.739074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:07.734364    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:07.739074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:07.735999    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx" event={"ID":"35da5897-e86e-4951-8199-1f50f4d6b2e4","Type":"ContainerStarted","Data":"5a7eafb8deaf525c189efa3b91e8ccf4cf1cae4da01a9b8f6f9d2f38cc592610"}
Nov 03 19:38:07.808112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:07.807262    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7" podStartSLOduration=2.143584212 podStartE2EDuration="7.807238278s" totalImagesPullingTime="5.663654066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:38:07.770570402 +0000 UTC m=+688.678592355" watchObservedRunningTime="2026-11-03 19:38:07.807238278 +0000 UTC m=+688.715260217"
Nov 03 19:38:07.808273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:07.808114    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz" podStartSLOduration=2.246142557 podStartE2EDuration="2.808101435s" totalImagesPullingTime="561.958878ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:38:07.80463052 +0000 UTC m=+688.712652473" watchObservedRunningTime="2026-11-03 19:38:07.808101435 +0000 UTC m=+688.716123380"
Nov 03 19:38:08.797334 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.797111    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4" event={"ID":"7ce60e5f-3d38-4e72-b857-bbdb675db1f5","Type":"ContainerStarted","Data":"ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9"}
Nov 03 19:38:08.797909 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.797889    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:38:08.800059 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.800027    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp" event={"ID":"b6c6d13b-5904-4d07-8534-4b724313bb79","Type":"ContainerStarted","Data":"9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8"}
Nov 03 19:38:08.800210 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.800199    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:38:08.803119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.802279    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx" event={"ID":"35da5897-e86e-4951-8199-1f50f4d6b2e4","Type":"ContainerStarted","Data":"0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e"}
Nov 03 19:38:08.803119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.802906    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:08.820357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.820322    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx" event={"ID":"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac","Type":"ContainerStarted","Data":"495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260"}
Nov 03 19:38:08.820516 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.820383    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:38:08.862065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.858771    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4" podStartSLOduration=2.443264748 podStartE2EDuration="8.858752786s" totalImagesPullingTime="6.415488038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:38:08.82375893 +0000 UTC m=+689.731780871" watchObservedRunningTime="2026-11-03 19:38:08.858752786 +0000 UTC m=+689.766774725"
Nov 03 19:38:08.898313 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.896362    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp" podStartSLOduration=2.611838805 podStartE2EDuration="8.896342777s" totalImagesPullingTime="6.284503972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:38:08.864342791 +0000 UTC m=+689.772364732" watchObservedRunningTime="2026-11-03 19:38:08.896342777 +0000 UTC m=+689.804364717"
Nov 03 19:38:08.918069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.918000    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx" podStartSLOduration=2.678242144 podStartE2EDuration="6.917985693s" totalImagesPullingTime="4.239743549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:38:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:38:08.897735437 +0000 UTC m=+689.805757378" watchObservedRunningTime="2026-11-03 19:38:08.917985693 +0000 UTC m=+689.826007632"
Nov 03 19:38:08.918223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:08.918201    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx" podStartSLOduration=2.401741141 podStartE2EDuration="2.918193325s" totalImagesPullingTime="516.452184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:38:08.917299279 +0000 UTC m=+689.825321235" watchObservedRunningTime="2026-11-03 19:38:08.918193325 +0000 UTC m=+689.826215263"
Nov 03 19:38:09.598535 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:09.598442    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:38:09.826083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:09.824532    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m" event={"ID":"82b74e21-57a7-489f-abe4-9814cf683aab","Type":"ContainerStarted","Data":"92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e"}
Nov 03 19:38:09.827997 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:09.827675    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:38:09.879169 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:09.856029    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m" podStartSLOduration=2.4572356539999998 podStartE2EDuration="8.856010012s" totalImagesPullingTime="6.398774358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:38:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:38:09.84536762 +0000 UTC m=+690.753389552" watchObservedRunningTime="2026-11-03 19:38:09.856010012 +0000 UTC m=+690.764031953"
Nov 03 19:38:11.724876 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:11.724477    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:38:12.575064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:12.569170    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:38:12.575064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:12.569270    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:38:14.833915 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:14.833826    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:38:14.834483 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:14.834462    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:38:16.920317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:16.920265    5051 generic.go:356] "Generic (PLEG): container finished" podID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerID="a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364" exitCode=0
Nov 03 19:38:16.920856 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:16.920321    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" event={"ID":"332ebb89-3bbd-47f3-a47e-33138a5bc16e","Type":"ContainerDied","Data":"a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364"}
Nov 03 19:38:16.920856 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:16.920850    5051 scope.go:122] "RemoveContainer" containerID="a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364"
Nov 03 19:38:16.922647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:16.922610    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:16.922753 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:16.922704    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:16.922814 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:16.922751    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:17.930309 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:17.930215    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" event={"ID":"332ebb89-3bbd-47f3-a47e-33138a5bc16e","Type":"ContainerStarted","Data":"ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8"}
Nov 03 19:38:17.932182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:17.932154    5051 generic.go:356] "Generic (PLEG): container finished" podID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerID="cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927" exitCode=0
Nov 03 19:38:17.932308 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:17.932193    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" event={"ID":"e2ae4144-2795-4fee-b8c7-9a5859c55d35","Type":"ContainerDied","Data":"cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927"}
Nov 03 19:38:17.932549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:17.932534    5051 scope.go:122] "RemoveContainer" containerID="cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927"
Nov 03 19:38:17.934036 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:17.934003    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:17.934187 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:17.934104    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:17.934187 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:17.934156    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:18.474370 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:18.474334    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:38:18.474581 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:18.474538    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 03 19:38:18.474662 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:18.474576    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 03 19:38:18.960123 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:18.941078    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" event={"ID":"e2ae4144-2795-4fee-b8c7-9a5859c55d35","Type":"ContainerStarted","Data":"5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef"}
Nov 03 19:38:19.098099 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:19.097021    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:38:19.098099 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:19.097180    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 03 19:38:19.098099 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:19.097223    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 03 19:38:19.476839 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:19.476404    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 03 19:38:19.476839 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:19.476460    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 03 19:38:19.828953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:19.828873    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:38:19.830116 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:19.830094    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:38:19.830256 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:19.830166    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:38:19.832154 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:19.832136    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:38:19.914424 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:19.913722    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:38:20.114875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:20.111604    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 03 19:38:20.114875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:20.111658    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 03 19:38:20.482114 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:20.477469    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 03 19:38:20.482114 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:20.477524    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 03 19:38:21.106068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:21.093789    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 03 19:38:21.106068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:21.093845    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 03 19:38:21.497089 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:21.496736    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 03 19:38:21.497089 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:21.496795    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 03 19:38:21.865119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:21.864201    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:38:22.104067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:22.096651    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 03 19:38:22.104067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:22.096703    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 03 19:38:22.483871 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:22.483433    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 03 19:38:22.483871 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:22.483490    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 03 19:38:23.087340 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.087256    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 03 19:38:23.087340 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.087310    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 03 19:38:23.102429 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.102392    5051 generic.go:356] "Generic (PLEG): container finished" podID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerID="4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf" exitCode=0
Nov 03 19:38:23.102603 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.102459    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" event={"ID":"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9","Type":"ContainerDied","Data":"4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf"}
Nov 03 19:38:23.102918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.102900    5051 scope.go:122] "RemoveContainer" containerID="4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf"
Nov 03 19:38:23.104266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.104246    5051 generic.go:356] "Generic (PLEG): container finished" podID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerID="ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237" exitCode=0
Nov 03 19:38:23.104381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.104333    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:23.104381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.104367    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" event={"ID":"bd2e2038-3d81-4246-8ef8-c29bfb7e8942","Type":"ContainerDied","Data":"ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237"}
Nov 03 19:38:23.104498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.104417    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:23.104498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.104465    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:23.104670 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.104654    5051 scope.go:122] "RemoveContainer" containerID="ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237"
Nov 03 19:38:23.105817 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.105792    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:23.105900 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.105849    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:23.105900 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:23.105881    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:38:24.090062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.089464    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Nov 03 19:38:24.090062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.089540    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Nov 03 19:38:24.138080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.136904    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" event={"ID":"bd2e2038-3d81-4246-8ef8-c29bfb7e8942","Type":"ContainerStarted","Data":"6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f"}
Nov 03 19:38:24.162101 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.161278    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" event={"ID":"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9","Type":"ContainerStarted","Data":"72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b"}
Nov 03 19:38:24.166556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.165404    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:38:24.166556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.166476    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:38:24.173008 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.169230    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:38:24.194062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.192892    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:38:24.194363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.194335    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 03 19:38:24.194537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.194498    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 03 19:38:24.611230 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.611145    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:38:24.611853 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.611817    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 19:38:24.611989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:24.611889    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 19:38:25.110076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:25.109212    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:38:25.110076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:25.109801    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:38:25.111695 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:25.111252    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:38:25.200068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:25.199237    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 03 19:38:25.200068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:25.199290    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 03 19:38:25.608917 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:25.608879    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 19:38:25.609151 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:25.608939    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 19:38:26.196589 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:26.196551    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 03 19:38:26.197079 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:26.196613    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 03 19:38:26.507378 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:26.507336    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:38:26.607406 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:26.607371    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 19:38:26.607583 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:26.607427    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 19:38:27.182065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:27.181423    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 03 19:38:27.182065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:27.181485    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 03 19:38:27.512441 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:27.512404    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:38:27.607829 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:27.607792    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 19:38:27.608009 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:27.607850    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 19:38:28.188064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:28.187416    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 03 19:38:28.188064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:28.187476    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 03 19:38:28.607389 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:28.607354    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 19:38:28.607787 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:28.607407    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 19:38:29.184091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:29.184055    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Nov 03 19:38:29.184381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:29.184350    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Nov 03 19:38:29.607980 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:29.607942    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 19:38:29.608650 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:29.608004    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 19:38:30.184836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:30.184800    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:38:30.185827 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:30.185804    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:38:30.298332 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:30.298299    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:38:30.611254 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:30.611222    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:38:30.612406 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:30.612386    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:38:31.280639 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.278176    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"]
Nov 03 19:38:31.288357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.287571    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.288357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.287875    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"]
Nov 03 19:38:31.316203 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.316180    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-05e217\"/\"simulator-w5-05e217-kserve-self-signed-certs\""
Nov 03 19:38:31.316360 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.316338    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-05e217\"/\"default-dockercfg-p65gc\""
Nov 03 19:38:31.318123 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.317197    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-05e217\"/\"openshift-service-ca.crt\""
Nov 03 19:38:31.318123 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.317324    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-05e217\"/\"kube-root-ca.crt\""
Nov 03 19:38:31.414943 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.414903    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-tmp-dir\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.415177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.415010    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-dshm\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.415177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.415084    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e112a60-5672-426a-8788-810b9cab54a1-tls-certs\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.415177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.415128    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5kkg\" (UniqueName: \"kubernetes.io/projected/1e112a60-5672-426a-8788-810b9cab54a1-kube-api-access-h5kkg\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.415359 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.415179    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-model-cache\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.415359 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.415218    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-home\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.516076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.516027    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-home\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.516270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.516149    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-tmp-dir\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.516270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.516201    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-dshm\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.516270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.516221    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e112a60-5672-426a-8788-810b9cab54a1-tls-certs\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.516270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.516247    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5kkg\" (UniqueName: \"kubernetes.io/projected/1e112a60-5672-426a-8788-810b9cab54a1-kube-api-access-h5kkg\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.516476 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.516331    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-model-cache\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.516563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.516532    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-home\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.516619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.516540    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-tmp-dir\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.516691 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.516673    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-model-cache\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.518433 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.518409    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-dshm\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.518716 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.518696    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e112a60-5672-426a-8788-810b9cab54a1-tls-certs\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.528406 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.528381    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5kkg\" (UniqueName: \"kubernetes.io/projected/1e112a60-5672-426a-8788-810b9cab54a1-kube-api-access-h5kkg\") pod \"simulator-w5-05e217-kserve-688bc8c868-jz8s8\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") " pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.624171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.624104    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:31.901598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:31.901568    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"]
Nov 03 19:38:31.902528 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:38:31.902489    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e112a60_5672_426a_8788_810b9cab54a1.slice/crio-be65dc9a39717ebe4f6160fcbf0102efb50a3d3d534f6eb341c490213810b3a2 WatchSource:0}: Error finding container be65dc9a39717ebe4f6160fcbf0102efb50a3d3d534f6eb341c490213810b3a2: Status 404 returned error can't find the container with id be65dc9a39717ebe4f6160fcbf0102efb50a3d3d534f6eb341c490213810b3a2
Nov 03 19:38:32.236996 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:32.236960    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" event={"ID":"1e112a60-5672-426a-8788-810b9cab54a1","Type":"ContainerStarted","Data":"7b5a14a8cc745ebaf81b087822691f35c56c0573de5983a18bde35b1990518d4"}
Nov 03 19:38:32.237147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:32.237005    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" event={"ID":"1e112a60-5672-426a-8788-810b9cab54a1","Type":"ContainerStarted","Data":"be65dc9a39717ebe4f6160fcbf0102efb50a3d3d534f6eb341c490213810b3a2"}
Nov 03 19:38:32.260216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:32.260168    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podStartSLOduration=1.260155195 podStartE2EDuration="1.260155195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:38:32.259562681 +0000 UTC m=+713.167584621" watchObservedRunningTime="2026-11-03 19:38:32.260155195 +0000 UTC m=+713.168177133"
Nov 03 19:38:32.625183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:32.625109    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:32.767390 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:38:32.767356    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:38:41.624898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:41.624868    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:41.638162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:41.638136    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:42.305173 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:42.305143    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:38:45.520809 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:45.520775    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:38:47.270585 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.270546    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"]
Nov 03 19:38:47.275963 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.275938    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.283216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.283194    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-ef8f84\"/\"openshift-service-ca.crt\""
Nov 03 19:38:47.284473 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.284458    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-ef8f84\"/\"kube-root-ca.crt\""
Nov 03 19:38:47.286171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.286152    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ef8f84\"/\"premium-w0-ef8f84-kserve-self-signed-certs\""
Nov 03 19:38:47.286267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.286215    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ef8f84\"/\"default-dockercfg-h9vlg\""
Nov 03 19:38:47.289601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.289576    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"]
Nov 03 19:38:47.296954 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.296930    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-home\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.297090 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.296961    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tmp-dir\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.297090 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.297003    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqz5j\" (UniqueName: \"kubernetes.io/projected/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-kube-api-access-vqz5j\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.297222 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.297109    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-dshm\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.297222 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.297171    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-model-cache\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.297222 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.297216    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tls-certs\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.398541 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.398503    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-home\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.398541 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.398547    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tmp-dir\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.398761 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.398582    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqz5j\" (UniqueName: \"kubernetes.io/projected/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-kube-api-access-vqz5j\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.398761 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.398614    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-dshm\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.398761 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.398642    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-model-cache\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.398761 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.398690    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tls-certs\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.399028 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.399003    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tmp-dir\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.399028 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.399019    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-model-cache\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.399153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.399075    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-home\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.401230 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.401195    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-dshm\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.401714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.401688    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tls-certs\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.419428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.413909    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqz5j\" (UniqueName: \"kubernetes.io/projected/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-kube-api-access-vqz5j\") pod \"premium-w0-ef8f84-kserve-7b7b76f977-djsqn\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") " pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.588369 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.588289    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:47.862905 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:47.862869    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"]
Nov 03 19:38:47.864107 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:38:47.864078    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ef91b29_e8d8_4003_9a0b_3f5f68d07a11.slice/crio-13ddeb7ea07a9a124ba1d30afdfc3b1e854cf1679dc09d7dddec70fb809ec3e6 WatchSource:0}: Error finding container 13ddeb7ea07a9a124ba1d30afdfc3b1e854cf1679dc09d7dddec70fb809ec3e6: Status 404 returned error can't find the container with id 13ddeb7ea07a9a124ba1d30afdfc3b1e854cf1679dc09d7dddec70fb809ec3e6
Nov 03 19:38:48.329915 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:48.329878    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" event={"ID":"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11","Type":"ContainerStarted","Data":"483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a"}
Nov 03 19:38:48.330305 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:48.329928    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" event={"ID":"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11","Type":"ContainerStarted","Data":"13ddeb7ea07a9a124ba1d30afdfc3b1e854cf1679dc09d7dddec70fb809ec3e6"}
Nov 03 19:38:48.353279 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:48.353209    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podStartSLOduration=1.353197139 podStartE2EDuration="1.353197139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:38:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:38:48.35073321 +0000 UTC m=+729.258755166" watchObservedRunningTime="2026-11-03 19:38:48.353197139 +0000 UTC m=+729.261219078"
Nov 03 19:38:48.588862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:48.588780    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:56.412501 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.412463    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"]
Nov 03 19:38:56.417452 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.417427    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.424834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.424134    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-d8a4ea\"/\"kube-root-ca.crt\""
Nov 03 19:38:56.424834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.424521    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-d8a4ea\"/\"openshift-service-ca.crt\""
Nov 03 19:38:56.424834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.424731    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-d8a4ea\"/\"default-dockercfg-ckb6m\""
Nov 03 19:38:56.425584 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.425556    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"]
Nov 03 19:38:56.425740 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.425601    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-d8a4ea\"/\"premium-w4-d8a4ea-kserve-self-signed-certs\""
Nov 03 19:38:56.514192 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.514157    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-home\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.514350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.514211    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88bpg\" (UniqueName: \"kubernetes.io/projected/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-kube-api-access-88bpg\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.514350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.514316    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tmp-dir\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.514424 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.514356    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tls-certs\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.514424 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.514375    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-model-cache\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.514526 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.514498    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-dshm\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.615558 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.615521    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88bpg\" (UniqueName: \"kubernetes.io/projected/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-kube-api-access-88bpg\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.615738 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.615574    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tmp-dir\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.615738 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.615619    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tls-certs\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.615738 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.615642    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-model-cache\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.615738 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.615688    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-dshm\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.615946 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.615793    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-home\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.616104 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.616081    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tmp-dir\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.616194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.616124    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-model-cache\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.616250 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.616232    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-home\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.617958 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.617936    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-dshm\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.618256 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.618239    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tls-certs\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.628900 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.628874    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88bpg\" (UniqueName: \"kubernetes.io/projected/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-kube-api-access-88bpg\") pod \"premium-w4-d8a4ea-kserve-5bb684988c-vqktq\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") " pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:56.734637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:56.734603    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:57.023804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:57.023777    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"]
Nov 03 19:38:57.025659 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:38:57.025630    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe6fda08_cc2c_48ed_a5bf_8dbc16810ab1.slice/crio-6d97b709be1b557c90df710de8d1e457a4397325f35841bd65c53718304b82be WatchSource:0}: Error finding container 6d97b709be1b557c90df710de8d1e457a4397325f35841bd65c53718304b82be: Status 404 returned error can't find the container with id 6d97b709be1b557c90df710de8d1e457a4397325f35841bd65c53718304b82be
Nov 03 19:38:57.387548 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:57.387463    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" event={"ID":"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1","Type":"ContainerStarted","Data":"d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f"}
Nov 03 19:38:57.387548 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:57.387505    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" event={"ID":"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1","Type":"ContainerStarted","Data":"6d97b709be1b557c90df710de8d1e457a4397325f35841bd65c53718304b82be"}
Nov 03 19:38:57.420995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:57.420947    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podStartSLOduration=1.420932263 podStartE2EDuration="1.420932263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:38:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:38:57.419428442 +0000 UTC m=+738.327450392" watchObservedRunningTime="2026-11-03 19:38:57.420932263 +0000 UTC m=+738.328954206"
Nov 03 19:38:57.588715 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:57.588678    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:57.601729 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:57.601702    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:38:57.734860 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:57.734837    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:38:58.404414 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:38:58.404383    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:39:01.101147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:01.101114    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:39:02.767825 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:39:02.767791    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:39:06.735507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:06.735476    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:39:06.747684 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:06.747659    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:39:07.461771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:07.461746    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:39:24.615495 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.615422    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"]
Nov 03 19:39:24.619234 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.619214    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.627094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.626981    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-435bd8-openshift-default-dockercfg-zznl2\""
Nov 03 19:39:24.632686 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.632657    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"]
Nov 03 19:39:24.707736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.707695    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-token\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.707736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.707736    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb5zt\" (UniqueName: \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-kube-api-access-rb5zt\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.707939 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.707809    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d291fb3-9492-4b4a-9495-e55d85d7480d-istiod-ca-cert\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.707939 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.707917    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-envoy\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.708026 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.708014    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-data\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.708167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.708059    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.708167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.708085    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-certs\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.708243 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.708204    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-podinfo\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.708276 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.708250    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-credential-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.801079 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.801030    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"]
Nov 03 19:39:24.804884 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.804862    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:24.809481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.809460    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-token\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.809584 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.809495    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb5zt\" (UniqueName: \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-kube-api-access-rb5zt\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.809584 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.809547    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d291fb3-9492-4b4a-9495-e55d85d7480d-istiod-ca-cert\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.809584 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.809575    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-envoy\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.809761 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.809663    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-data\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.809761 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.809700    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.809761 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.809748    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-certs\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.809913 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.809800    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-podinfo\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.809913 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.809869    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-credential-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.810174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.810057    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-data\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.810174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.810127    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-certs\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.810174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.810158    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.810358 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.810271    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-credential-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.810358 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.810313    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d291fb3-9492-4b4a-9495-e55d85d7480d-istiod-ca-cert\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.810706 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.810682    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-2c3fc4-openshift-default-dockercfg-f8v5n\""
Nov 03 19:39:24.812204 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.812182    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-envoy\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.812978 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.812956    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-podinfo\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.819620 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.819599    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"]
Nov 03 19:39:24.826870 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.826827    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-token\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.830539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.830491    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb5zt\" (UniqueName: \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-kube-api-access-rb5zt\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:24.910651 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.910576    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:24.910651 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.910629    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:24.910853 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.910758    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:24.910853 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.910793    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9fn6\" (UniqueName: \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-kube-api-access-q9fn6\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:24.910916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.910878    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:24.910961 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.910913    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:24.910961 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.910929    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:24.911061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.910967    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:24.911061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.911027    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:24.934215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:24.934188    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:25.013126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.013089    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.013273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.013160    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.013273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.013191    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.013273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.013221    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.013273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.013267    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.013485 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.013399    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.013485 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.013473    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.013586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.013562    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.013642 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.013595    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9fn6\" (UniqueName: \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-kube-api-access-q9fn6\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.014432 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.014402    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.014809 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.014785    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.015439 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.015414    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.015938 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.015913    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.016530 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.016503    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.021619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.021598    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.023788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.023764    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.026347 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.026326    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9fn6\" (UniqueName: \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-kube-api-access-q9fn6\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.030021 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.029994    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.097451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.097423    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"]
Nov 03 19:39:25.102543 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.102525    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.107092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.107070    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-d8a4ea-openshift-default-dockercfg-wkq5q\""
Nov 03 19:39:25.114462 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.114439    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"]
Nov 03 19:39:25.115015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.114994    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.115155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.115033    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.115155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.115076    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.115280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.115193    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.115280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.115224    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.115280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.115261    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.115442 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.115320    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.115442 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.115348    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75g4g\" (UniqueName: \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-kube-api-access-75g4g\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.115442 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.115371    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.154227 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.154200    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:25.220826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.219226    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"]
Nov 03 19:39:25.220826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.219486    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.220826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.219542    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.220826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.219572    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.220826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.219602    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.220826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.219631    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.220826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.219662    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.220826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.219743    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.220826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.219783    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75g4g\" (UniqueName: \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-kube-api-access-75g4g\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.220826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.219844    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.222187 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.222121    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.222789 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.222567    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.223012 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.222989    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.223995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.223967    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.225797 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.225780    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.226736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.226710    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.228790 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.228684    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.229576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.229539    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:25.229687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.229619    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:25.229687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.229658    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:25.237029 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.237003    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.238408 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.238386    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75g4g\" (UniqueName: \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-kube-api-access-75g4g\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.417270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.417183    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:25.438672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.438644    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"]
Nov 03 19:39:25.440574 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:39:25.440552    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52e2dd2_1554_47a3_a7c9_5a7a78fe2730.slice/crio-36f0e0e210f38198d443265347f3820ff0def869686fb9bdbbe1d9c1ff019941 WatchSource:0}: Error finding container 36f0e0e210f38198d443265347f3820ff0def869686fb9bdbbe1d9c1ff019941: Status 404 returned error can't find the container with id 36f0e0e210f38198d443265347f3820ff0def869686fb9bdbbe1d9c1ff019941
Nov 03 19:39:25.443395 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.443363    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:25.443482 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.443438    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:25.443550 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.443484    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:25.570266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.570228    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" event={"ID":"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730","Type":"ContainerStarted","Data":"d265ca7b3744977dd923ff9b62d094379a291cfea0cd4318203a814d655116fe"}
Nov 03 19:39:25.570406 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.570300    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" event={"ID":"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730","Type":"ContainerStarted","Data":"36f0e0e210f38198d443265347f3820ff0def869686fb9bdbbe1d9c1ff019941"}
Nov 03 19:39:25.579079 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.572299    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" event={"ID":"7d291fb3-9492-4b4a-9495-e55d85d7480d","Type":"ContainerStarted","Data":"2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03"}
Nov 03 19:39:25.579079 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.572339    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" event={"ID":"7d291fb3-9492-4b4a-9495-e55d85d7480d","Type":"ContainerStarted","Data":"24ae5d04fcdf2faea4a489dbf4e5ab136a165e385e0f73975755006b6e1f9257"}
Nov 03 19:39:25.597495 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.597431    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podStartSLOduration=1.597412425 podStartE2EDuration="1.597412425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:39:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:39:25.594222601 +0000 UTC m=+766.502244565" watchObservedRunningTime="2026-11-03 19:39:25.597412425 +0000 UTC m=+766.505434369"
Nov 03 19:39:25.623773 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.623706    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" podStartSLOduration=1.623688056 podStartE2EDuration="1.623688056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:39:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:39:25.62211169 +0000 UTC m=+766.530133657" watchObservedRunningTime="2026-11-03 19:39:25.623688056 +0000 UTC m=+766.531709998"
Nov 03 19:39:25.719361 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.717937    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"]
Nov 03 19:39:25.723082 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:39:25.723021    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12bfd9b_aae3_489d_a44e_70dab3da12ae.slice/crio-fee8f7fbf4b38db915fa6cc8ab1b45b65f7785e48abafc7382e748efc0a96e1e WatchSource:0}: Error finding container fee8f7fbf4b38db915fa6cc8ab1b45b65f7785e48abafc7382e748efc0a96e1e: Status 404 returned error can't find the container with id fee8f7fbf4b38db915fa6cc8ab1b45b65f7785e48abafc7382e748efc0a96e1e
Nov 03 19:39:25.725649 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.725614    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:25.725730 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.725697    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:25.725779 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.725739    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:25.934421 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:25.934393    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:26.162124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:26.161105    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:26.162124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:26.161927    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 03 19:39:26.162124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:26.161976    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 03 19:39:26.596072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:26.595019    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" event={"ID":"f12bfd9b-aae3-489d-a44e-70dab3da12ae","Type":"ContainerStarted","Data":"3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc"}
Nov 03 19:39:26.596072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:26.595078    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" event={"ID":"f12bfd9b-aae3-489d-a44e-70dab3da12ae","Type":"ContainerStarted","Data":"fee8f7fbf4b38db915fa6cc8ab1b45b65f7785e48abafc7382e748efc0a96e1e"}
Nov 03 19:39:26.628377 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:26.628285    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" podStartSLOduration=1.628231785 podStartE2EDuration="1.628231785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:39:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:39:26.620935163 +0000 UTC m=+767.528957116" watchObservedRunningTime="2026-11-03 19:39:26.628231785 +0000 UTC m=+767.536253723"
Nov 03 19:39:26.935808 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:26.935729    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 03 19:39:26.936004 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:26.935981    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 03 19:39:27.164065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:27.162544    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 03 19:39:27.164065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:27.162609    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 03 19:39:27.418506 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:27.418466    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:27.419857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:27.419833    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 19:39:27.419942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:27.419882    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 19:39:27.934511 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:27.934482    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 03 19:39:27.934971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:27.934949    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 03 19:39:28.164065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:28.163114    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 03 19:39:28.164065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:28.163170    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 03 19:39:28.418297 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:28.418255    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 19:39:28.418521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:28.418322    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 19:39:28.935934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:28.935667    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 03 19:39:28.935934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:28.935747    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 03 19:39:29.164193 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:29.164157    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 03 19:39:29.164449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:29.164420    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 03 19:39:29.231878 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:29.231848    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:39:29.418406 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:29.418366    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 19:39:29.418618 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:29.418418    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 19:39:29.934755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:29.934718    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 03 19:39:29.935066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:29.935015    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 03 19:39:30.166121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:30.166085    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 03 19:39:30.166717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:30.166687    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 03 19:39:30.422069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:30.420567    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 19:39:30.422069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:30.420630    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 19:39:30.935484 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:30.935447    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 03 19:39:30.935776 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:30.935751    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 03 19:39:31.158491 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:31.158449    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 03 19:39:31.159886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:31.159851    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 03 19:39:31.463065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:31.440325    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 19:39:31.463065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:31.440385    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 19:39:31.934650 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:31.934553    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 03 19:39:31.934650 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:31.934603    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 03 19:39:32.155523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:32.155487    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 03 19:39:32.155857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:32.155828    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 03 19:39:32.418291 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:32.418253    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 19:39:32.418643 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:32.418602    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 19:39:32.771935 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:39:32.771896    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:39:33.508229 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:33.508192    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:33.509212 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:33.509179    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:33.510145 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:33.510124    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:39:34.086088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:34.084768    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:34.086603 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:34.086168    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:34.101942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:34.098630    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:39:34.189296 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:34.189257    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:39:34.189456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:34.189324    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:39:34.190349 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:34.190325    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:34.190663 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:34.190642    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:34.191558 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:34.191539    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:39:55.068990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.068954    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"]
Nov 03 19:39:55.073082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.073026    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.102270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.102246    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"]
Nov 03 19:39:55.204259 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.204229    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.204446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.204270    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.204446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.204376    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tq5d\" (UniqueName: \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-kube-api-access-6tq5d\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.204634 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.204481    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.204634 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.204527    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf903bff-eb4a-436e-af4c-9bdae7c21698-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.204634 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.204565    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.204750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.204620    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.204750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.204682    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.204852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.204831    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.305909 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.305880    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.305938    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.305957    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.306011    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tq5d\" (UniqueName: \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-kube-api-access-6tq5d\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.306032    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.306082    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf903bff-eb4a-436e-af4c-9bdae7c21698-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306405 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.306122    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306405 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.306145    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306405 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.306179    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306587 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.306532    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306646 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.306613    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306646 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.306637    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.306850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.306834    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.307036 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.307014    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf903bff-eb4a-436e-af4c-9bdae7c21698-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.308317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.308293    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.308483 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.308467    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.318540 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.318503    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.318644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.318550    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tq5d\" (UniqueName: \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-kube-api-access-6tq5d\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.388954 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.388902    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:55.671519 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.671485    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"]
Nov 03 19:39:55.672391 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:39:55.672349    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf903bff_eb4a_436e_af4c_9bdae7c21698.slice/crio-f950147284af93bcd61447746e02c371d4b4096fee52379bc085b621ef041a5a WatchSource:0}: Error finding container f950147284af93bcd61447746e02c371d4b4096fee52379bc085b621ef041a5a: Status 404 returned error can't find the container with id f950147284af93bcd61447746e02c371d4b4096fee52379bc085b621ef041a5a
Nov 03 19:39:55.675329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.675290    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:55.675431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.675381    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:55.675481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.675426    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:39:55.849661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.849628    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" event={"ID":"bf903bff-eb4a-436e-af4c-9bdae7c21698","Type":"ContainerStarted","Data":"885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa"}
Nov 03 19:39:55.849852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.849684    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" event={"ID":"bf903bff-eb4a-436e-af4c-9bdae7c21698","Type":"ContainerStarted","Data":"f950147284af93bcd61447746e02c371d4b4096fee52379bc085b621ef041a5a"}
Nov 03 19:39:55.875348 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:55.875299    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" podStartSLOduration=0.875286149 podStartE2EDuration="875.286149ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:39:55.872519981 +0000 UTC m=+796.780541942" watchObservedRunningTime="2026-11-03 19:39:55.875286149 +0000 UTC m=+796.783308088"
Nov 03 19:39:56.389648 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:56.389616    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:39:56.391653 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:56.391625    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 03 19:39:56.391779 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:56.391680    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 03 19:39:57.389623 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:57.389588    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 03 19:39:57.389804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:57.389642    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 03 19:39:58.393066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:58.391549    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 03 19:39:58.393066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:58.391621    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 03 19:39:59.394951 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:59.394452    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 03 19:39:59.394951 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:39:59.394538    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 03 19:40:00.389750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:00.389715    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 03 19:40:00.389952 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:00.389765    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 03 19:40:01.389939 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:01.389905    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 03 19:40:01.390376 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:01.389955    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 03 19:40:02.768752 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:40:02.768714    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:40:03.186176 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:03.186093    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:40:03.186544 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:03.186519    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:40:03.187526 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:03.187508    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:40:11.977978 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:11.977944    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:40:29.664189 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.664150    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"]
Nov 03 19:40:29.715121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.699222    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"]
Nov 03 19:40:29.715952 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.705189    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.737419 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.737389    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-tmp-dir\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.765526 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.765487    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d11a037-b776-43f9-9d65-fa507737d381-tls-certs\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.790077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.766259    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-dshm\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.790077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.766322    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-model-cache\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.790077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.765936    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-2c3fc4\"/\"kube-root-ca.crt\""
Nov 03 19:40:29.790077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.766468    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6ftf\" (UniqueName: \"kubernetes.io/projected/8d11a037-b776-43f9-9d65-fa507737d381-kube-api-access-p6ftf\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.790077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.766552    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-2c3fc4\"/\"openshift-service-ca.crt\""
Nov 03 19:40:29.790077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.766571    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-home\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.790077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.766472    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-2c3fc4\"/\"default-dockercfg-lhd72\""
Nov 03 19:40:29.790077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.766824    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-2c3fc4\"/\"distinct-w1-2c3fc4-kserve-self-signed-certs\""
Nov 03 19:40:29.923067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.922293    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6ftf\" (UniqueName: \"kubernetes.io/projected/8d11a037-b776-43f9-9d65-fa507737d381-kube-api-access-p6ftf\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.923067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.922394    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-home\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.923067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.922426    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-tmp-dir\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.923067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.922452    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d11a037-b776-43f9-9d65-fa507737d381-tls-certs\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.923067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.922577    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-dshm\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.923067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.922617    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-model-cache\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.923537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.923282    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-model-cache\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.935172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.923838    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-home\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:29.952335 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.951206    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-tmp-dir\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:30.005115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:29.994862    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d11a037-b776-43f9-9d65-fa507737d381-tls-certs\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:30.025116 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:30.017316    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-dshm\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:30.042078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:30.033583    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6ftf\" (UniqueName: \"kubernetes.io/projected/8d11a037-b776-43f9-9d65-fa507737d381-kube-api-access-p6ftf\") pod \"distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") " pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:30.143564 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:30.143531    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:30.320232 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:30.320188    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:40:30.493067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:30.492938    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:40:31.204636 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:31.204546    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"]
Nov 03 19:40:31.665734 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:31.662837    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" event={"ID":"8d11a037-b776-43f9-9d65-fa507737d381","Type":"ContainerStarted","Data":"fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385"}
Nov 03 19:40:31.665734 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:31.662898    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" event={"ID":"8d11a037-b776-43f9-9d65-fa507737d381","Type":"ContainerStarted","Data":"2331981ddfa901b741a6bf5a082db03e053df0939107eb400b01539eb1ad04f1"}
Nov 03 19:40:31.695159 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:31.693987    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podStartSLOduration=2.69396976 podStartE2EDuration="2.69396976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:40:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:40:31.692242001 +0000 UTC m=+832.600263957" watchObservedRunningTime="2026-11-03 19:40:31.69396976 +0000 UTC m=+832.601991700"
Nov 03 19:40:32.151063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:32.149790    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:32.767999 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:40:32.767957    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:40:39.556168 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.556123    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"]
Nov 03 19:40:39.564293 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.564267    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.591794 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.591757    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"]
Nov 03 19:40:39.602907 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.602877    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.603088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.602919    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.603088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.603076    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.603205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.603115    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq866\" (UniqueName: \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-kube-api-access-vq866\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.603205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.603152    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.603302 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.603280    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/951771a5-2df2-4a73-b5a1-34dd546f9c46-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.603386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.603370    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.603474 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.603452    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.603514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.603503    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.704702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.704676    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.704865 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.704774    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.704865 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.704798    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.704865 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.704849    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.705026 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.704878    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq866\" (UniqueName: \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-kube-api-access-vq866\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.705026 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.704990    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.705198 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.705095    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/951771a5-2df2-4a73-b5a1-34dd546f9c46-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.705198 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.705174    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.705306 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.705209    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.705306 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.705274    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.705387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.705333    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.705421 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.705388    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.705583 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.705562    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.705736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.705711    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/951771a5-2df2-4a73-b5a1-34dd546f9c46-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.707312 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.707284    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.707417 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.707288    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.717188 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.717163    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.717330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.717313    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq866\" (UniqueName: \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-kube-api-access-vq866\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:39.885017 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:39.884948    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:40.150157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.150085    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:40.162246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.162220    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:40.166726 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.166703    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"]
Nov 03 19:40:40.167242 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:40:40.167220    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod951771a5_2df2_4a73_b5a1_34dd546f9c46.slice/crio-a1f898b9455a2ed900f61dffacdb90b006b7eaf1cc4fb2351484ae7f095a7f25 WatchSource:0}: Error finding container a1f898b9455a2ed900f61dffacdb90b006b7eaf1cc4fb2351484ae7f095a7f25: Status 404 returned error can't find the container with id a1f898b9455a2ed900f61dffacdb90b006b7eaf1cc4fb2351484ae7f095a7f25
Nov 03 19:40:40.169872 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.169843    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:40:40.169954 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.169904    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:40:40.169954 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.169932    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:40:40.722890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.722855    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" event={"ID":"951771a5-2df2-4a73-b5a1-34dd546f9c46","Type":"ContainerStarted","Data":"d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a"}
Nov 03 19:40:40.723265 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.722909    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" event={"ID":"951771a5-2df2-4a73-b5a1-34dd546f9c46","Type":"ContainerStarted","Data":"a1f898b9455a2ed900f61dffacdb90b006b7eaf1cc4fb2351484ae7f095a7f25"}
Nov 03 19:40:40.734424 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.734398    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:40:40.750609 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.750518    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" podStartSLOduration=1.7505006810000001 podStartE2EDuration="1.750500681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:40:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:40:40.748582765 +0000 UTC m=+841.656604730" watchObservedRunningTime="2026-11-03 19:40:40.750500681 +0000 UTC m=+841.658522620"
Nov 03 19:40:40.786130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.786095    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"]
Nov 03 19:40:40.793944 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.793878    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.811413 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.810915    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-05e217\"/\"distinct-w5-05e217-kserve-self-signed-certs\""
Nov 03 19:40:40.813016 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.812973    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"]
Nov 03 19:40:40.818658 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.818628    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-home\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.818776 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.818670    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tmp-dir\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.818900 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.818877    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-dshm\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.818948 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.818908    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-model-cache\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.819086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.819058    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wmbb\" (UniqueName: \"kubernetes.io/projected/bdc62bab-9a46-4aac-81ef-21735bcc8e43-kube-api-access-4wmbb\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.819358 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.819336    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tls-certs\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.885092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.885065    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:40.920675 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.920640    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tls-certs\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.920843 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.920706    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-home\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.920843 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.920752    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tmp-dir\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.920843 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.920813    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-dshm\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.920843 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.920833    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-model-cache\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.921116 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.920910    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmbb\" (UniqueName: \"kubernetes.io/projected/bdc62bab-9a46-4aac-81ef-21735bcc8e43-kube-api-access-4wmbb\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.921215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.921192    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tmp-dir\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.921273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.921257    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-home\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.921323 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.921289    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-model-cache\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.923007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.922986    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-dshm\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.923288 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.923270    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tls-certs\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:40.933982 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:40.933957    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wmbb\" (UniqueName: \"kubernetes.io/projected/bdc62bab-9a46-4aac-81ef-21735bcc8e43-kube-api-access-4wmbb\") pod \"distinct-w5-05e217-kserve-78f864878d-cvzj4\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") " pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:41.116773 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:41.116691    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:41.420195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:41.420169    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"]
Nov 03 19:40:41.420796 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:40:41.420773    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdc62bab_9a46_4aac_81ef_21735bcc8e43.slice/crio-fa1ca5ccbc5bd5ed9d2de2c83dc330edaed380da57036c1cfacf1cf84b9977da WatchSource:0}: Error finding container fa1ca5ccbc5bd5ed9d2de2c83dc330edaed380da57036c1cfacf1cf84b9977da: Status 404 returned error can't find the container with id fa1ca5ccbc5bd5ed9d2de2c83dc330edaed380da57036c1cfacf1cf84b9977da
Nov 03 19:40:41.738382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:41.738338    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" event={"ID":"bdc62bab-9a46-4aac-81ef-21735bcc8e43","Type":"ContainerStarted","Data":"14b7d9141c7bbac6d22f5fe862e731b04ade0dccc71da1881a4edcf3862b9fcd"}
Nov 03 19:40:41.738862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:41.738396    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" event={"ID":"bdc62bab-9a46-4aac-81ef-21735bcc8e43","Type":"ContainerStarted","Data":"fa1ca5ccbc5bd5ed9d2de2c83dc330edaed380da57036c1cfacf1cf84b9977da"}
Nov 03 19:40:41.785064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:41.784998    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podStartSLOduration=1.784983908 podStartE2EDuration="1.784983908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:40:41.783483546 +0000 UTC m=+842.691505484" watchObservedRunningTime="2026-11-03 19:40:41.784983908 +0000 UTC m=+842.693005847"
Nov 03 19:40:41.886977 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:41.886941    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 19:40:41.887145 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:41.886993    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 19:40:42.117113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:42.117009    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:42.885364 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:42.885325    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 19:40:42.885831 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:42.885380    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 19:40:43.308303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:43.308270    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:40:43.885533 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:43.885499    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 19:40:43.885890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:43.885552    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 19:40:44.895064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:44.890580    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 19:40:44.895064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:44.890655    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 19:40:45.902064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:45.886415    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 19:40:45.902064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:45.886478    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 19:40:46.886270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:46.886231    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 19:40:46.886444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:46.886293    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 19:40:47.977194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:47.977161    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:47.977836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:47.977812    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:47.979528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:47.979503    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:40:51.117363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:51.117274    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:51.129988 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:51.129965    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:51.938777 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:51.938730    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:40:52.985630 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:52.985597    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"]
Nov 03 19:40:52.990842 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:52.990824    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.001527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.001502    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-d8a4ea\"/\"distinct-2-w4-d8a4ea-kserve-self-signed-certs\""
Nov 03 19:40:53.004538 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.004510    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"]
Nov 03 19:40:53.141206 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.141161    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-model-cache\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.141377 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.141246    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-dshm\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.141377 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.141294    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-home\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.141488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.141383    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f3db017-2c11-424f-9acb-51cfa24a92c4-tls-certs\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.141488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.141441    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sztsx\" (UniqueName: \"kubernetes.io/projected/6f3db017-2c11-424f-9acb-51cfa24a92c4-kube-api-access-sztsx\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.141488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.141481    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-tmp-dir\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.243249 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.243169    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-model-cache\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.243249 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.243220    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-dshm\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.243431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.243258    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-home\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.243431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.243326    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f3db017-2c11-424f-9acb-51cfa24a92c4-tls-certs\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.243431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.243372    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sztsx\" (UniqueName: \"kubernetes.io/projected/6f3db017-2c11-424f-9acb-51cfa24a92c4-kube-api-access-sztsx\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.243534 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.243448    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-tmp-dir\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.243616 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.243599    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-model-cache\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.243724 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.243698    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-home\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.243838 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.243754    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-tmp-dir\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.245413 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.245393    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-dshm\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.245792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.245773    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f3db017-2c11-424f-9acb-51cfa24a92c4-tls-certs\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.258756 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.258730    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sztsx\" (UniqueName: \"kubernetes.io/projected/6f3db017-2c11-424f-9acb-51cfa24a92c4-kube-api-access-sztsx\") pod \"distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") " pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.303915 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.303883    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:53.582503 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.582473    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"]
Nov 03 19:40:53.582993 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:40:53.582963    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f3db017_2c11_424f_9acb_51cfa24a92c4.slice/crio-8cfd3f3fd467b759a7aeda4cfd333327939870c8c3da124c1a7f132724dab739 WatchSource:0}: Error finding container 8cfd3f3fd467b759a7aeda4cfd333327939870c8c3da124c1a7f132724dab739: Status 404 returned error can't find the container with id 8cfd3f3fd467b759a7aeda4cfd333327939870c8c3da124c1a7f132724dab739
Nov 03 19:40:53.942676 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.942588    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" event={"ID":"6f3db017-2c11-424f-9acb-51cfa24a92c4","Type":"ContainerStarted","Data":"4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04"}
Nov 03 19:40:53.942676 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.942633    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" event={"ID":"6f3db017-2c11-424f-9acb-51cfa24a92c4","Type":"ContainerStarted","Data":"8cfd3f3fd467b759a7aeda4cfd333327939870c8c3da124c1a7f132724dab739"}
Nov 03 19:40:53.964535 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:53.964486    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podStartSLOduration=1.964471796 podStartE2EDuration="1.964471796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:40:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:40:53.961443766 +0000 UTC m=+854.869465738" watchObservedRunningTime="2026-11-03 19:40:53.964471796 +0000 UTC m=+854.872493735"
Nov 03 19:40:54.304200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:54.304167    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:40:55.249857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.249824    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"]
Nov 03 19:40:55.253810 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.253790    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.275713 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.275687    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"]
Nov 03 19:40:55.370227 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.370192    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.370609 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.370261    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp5fg\" (UniqueName: \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-kube-api-access-zp5fg\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.370609 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.370382    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.370609 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.370421    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.370609 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.370460    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.370743 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.370629    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.370743 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.370664    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.370743 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.370684    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.370743 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.370707    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.471448 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.471407    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.471600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.471461    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp5fg\" (UniqueName: \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-kube-api-access-zp5fg\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.471600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.471497    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.471600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.471516    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.471600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.471544    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.471834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.471659    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.471834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.471692    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.471834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.471720    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.471834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.471757    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.472083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.471842    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.472083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.471922    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.472194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.472131    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.472381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.472362    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.472618 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.472595    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.473862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.473842    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.474327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.474310    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.491644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.491621    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.491823 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.491803    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp5fg\" (UniqueName: \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-kube-api-access-zp5fg\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.568376 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.568317    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:55.868249 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.868212    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"]
Nov 03 19:40:55.873381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.873341    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:40:55.873508 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.873431    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:40:55.873508 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.873479    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:40:55.963220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:55.963184    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" event={"ID":"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c","Type":"ContainerStarted","Data":"05a8908d24daa1475de700ccd4ec7d74649aa24cb0882c5ea3a958dc14201099"}
Nov 03 19:40:56.989444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:56.989393    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" event={"ID":"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c","Type":"ContainerStarted","Data":"09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4"}
Nov 03 19:40:57.026418 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:57.026357    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" podStartSLOduration=2.026334641 podStartE2EDuration="2.026334641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:40:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:40:57.023489897 +0000 UTC m=+857.931511848" watchObservedRunningTime="2026-11-03 19:40:57.026334641 +0000 UTC m=+857.934356582"
Nov 03 19:40:57.568828 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:57.568794    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:40:57.570831 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:57.570807    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 03 19:40:57.570938 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:57.570851    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 03 19:40:58.569542 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:58.569501    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 03 19:40:58.569940 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:58.569583    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 03 19:40:59.569571 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:59.569535    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 03 19:40:59.569971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:40:59.569594    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 03 19:41:00.569531 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:00.569496    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 03 19:41:00.569785 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:00.569549    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 03 19:41:01.568979 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:01.568948    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 03 19:41:01.569164 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:01.569003    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 03 19:41:02.568896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:02.568854    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 03 19:41:02.569346 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:02.568913    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 03 19:41:02.768813 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:41:02.768775    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:41:03.306050 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:03.306003    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:41:03.325540 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:03.325510    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:41:03.572986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:03.572895    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:41:03.573484 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:03.573327    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:41:03.574031 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:03.574010    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:41:04.059273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:04.059243    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:41:09.950773 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.950740    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"]
Nov 03 19:41:09.956368 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.956348    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:09.973325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.973299    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"]
Nov 03 19:41:09.982856 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.982823    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:09.982995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.982873    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d4704e5-7110-4e60-8864-05717066c219-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:09.983106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.983081    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:09.983181 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.983163    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:09.983242 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.983193    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqh6w\" (UniqueName: \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-kube-api-access-jqh6w\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:09.983303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.983259    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d4704e5-7110-4e60-8864-05717066c219-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:09.983361 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.983302    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:09.983410 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.983367    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:09.983467 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:09.983411    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.084587 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.084547    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.090199    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.090236    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqh6w\" (UniqueName: \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-kube-api-access-jqh6w\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.090307    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d4704e5-7110-4e60-8864-05717066c219-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.090350    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.090421    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.090468    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.090567    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.090605    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d4704e5-7110-4e60-8864-05717066c219-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.090657    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.091183    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.091424    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.091743    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.095754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.091812    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d4704e5-7110-4e60-8864-05717066c219-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.097993 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.097943    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.102827 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.102798    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.105446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.105421    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d4704e5-7110-4e60-8864-05717066c219-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.107788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.107757    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqh6w\" (UniqueName: \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-kube-api-access-jqh6w\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.273937 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.273903    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:10.565948 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.565919    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"]
Nov 03 19:41:10.567191 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:41:10.567164    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d4704e5_7110_4e60_8864_05717066c219.slice/crio-654dbcb15e99898de6836f481ad07163ee6ce019be4b1e445542165affd15c32 WatchSource:0}: Error finding container 654dbcb15e99898de6836f481ad07163ee6ce019be4b1e445542165affd15c32: Status 404 returned error can't find the container with id 654dbcb15e99898de6836f481ad07163ee6ce019be4b1e445542165affd15c32
Nov 03 19:41:10.569702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.569671    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:10.569788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.569730    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:10.569788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.569761    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:10.605520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:10.605494    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:41:11.112001 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:11.111963    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" event={"ID":"7d4704e5-7110-4e60-8864-05717066c219","Type":"ContainerStarted","Data":"e1111be2b9ba5609b16c755aecc6ebf6204b78f5552708b95075be7e7d10516b"}
Nov 03 19:41:11.112421 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:11.112023    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" event={"ID":"7d4704e5-7110-4e60-8864-05717066c219","Type":"ContainerStarted","Data":"654dbcb15e99898de6836f481ad07163ee6ce019be4b1e445542165affd15c32"}
Nov 03 19:41:11.170353 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:11.170290    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" podStartSLOduration=2.170270828 podStartE2EDuration="2.170270828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:11.160411908 +0000 UTC m=+872.068433870" watchObservedRunningTime="2026-11-03 19:41:11.170270828 +0000 UTC m=+872.078292766"
Nov 03 19:41:11.274411 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:11.274368    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:11.276151 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:11.276120    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 03 19:41:11.276285 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:11.276169    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 03 19:41:12.276504 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:12.276448    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 03 19:41:12.277006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:12.276527    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 03 19:41:13.279065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:13.278482    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 03 19:41:13.279065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:13.278566    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 03 19:41:14.275026 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:14.274996    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 03 19:41:14.275239 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:14.275066    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 03 19:41:15.275103 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:15.275034    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 03 19:41:15.275592 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:15.275152    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 03 19:41:16.277605 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:16.277567    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 03 19:41:16.278520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:16.278486    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 03 19:41:17.275276 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.275240    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 03 19:41:17.275456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.275304    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 03 19:41:17.651520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.651437    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"]
Nov 03 19:41:17.657736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.657713    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.677665 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.677634    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"]
Nov 03 19:41:17.681354 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.681332    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ef8f84\"/\"distinct-w0-ef8f84-kserve-self-signed-certs\""
Nov 03 19:41:17.690867 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.690843    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-tmp-dir\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.690990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.690882    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b5a03fb-7018-4e65-984a-719d0284ae34-tls-certs\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.690990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.690940    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-model-cache\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.690990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.690981    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw95v\" (UniqueName: \"kubernetes.io/projected/0b5a03fb-7018-4e65-984a-719d0284ae34-kube-api-access-xw95v\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.691189 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.691118    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-home\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.691262 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.691193    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-dshm\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.792769 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.792729    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-home\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.793081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.792810    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-dshm\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.793081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.792931    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-tmp-dir\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.793081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.792966    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b5a03fb-7018-4e65-984a-719d0284ae34-tls-certs\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.793300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.793136    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-model-cache\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.793300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.793215    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-home\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.793300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.793215    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw95v\" (UniqueName: \"kubernetes.io/projected/0b5a03fb-7018-4e65-984a-719d0284ae34-kube-api-access-xw95v\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.793526 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.793500    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-tmp-dir\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.793658 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.793550    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-model-cache\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.795263 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.795239    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-dshm\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.795785 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.795755    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b5a03fb-7018-4e65-984a-719d0284ae34-tls-certs\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.811791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.811725    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw95v\" (UniqueName: \"kubernetes.io/projected/0b5a03fb-7018-4e65-984a-719d0284ae34-kube-api-access-xw95v\") pod \"distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") " pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:17.995601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:17.995572    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:18.279099 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:18.279013    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:18.279328 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:18.279308    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:18.279886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:18.279867    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:41:18.285072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:18.285050    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"]
Nov 03 19:41:18.285892 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:41:18.285872    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b5a03fb_7018_4e65_984a_719d0284ae34.slice/crio-3678c026bbc70afc5c2b8867cb0d91178a60476138efc076c2a646a4014089fc WatchSource:0}: Error finding container 3678c026bbc70afc5c2b8867cb0d91178a60476138efc076c2a646a4014089fc: Status 404 returned error can't find the container with id 3678c026bbc70afc5c2b8867cb0d91178a60476138efc076c2a646a4014089fc
Nov 03 19:41:19.196462 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.196429    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" event={"ID":"0b5a03fb-7018-4e65-984a-719d0284ae34","Type":"ContainerStarted","Data":"cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9"}
Nov 03 19:41:19.196846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.196473    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" event={"ID":"0b5a03fb-7018-4e65-984a-719d0284ae34","Type":"ContainerStarted","Data":"3678c026bbc70afc5c2b8867cb0d91178a60476138efc076c2a646a4014089fc"}
Nov 03 19:41:19.218495 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.218447    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podStartSLOduration=2.218433054 podStartE2EDuration="2.218433054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:19.215753498 +0000 UTC m=+880.123775448" watchObservedRunningTime="2026-11-03 19:41:19.218433054 +0000 UTC m=+880.126455005"
Nov 03 19:41:19.651702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.651663    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"]
Nov 03 19:41:19.666633 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.665403    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.671223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.671193    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-435bd8\"/\"default-dockercfg-nt8gv\""
Nov 03 19:41:19.672525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.672498    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-435bd8\"/\"distinct-2-w2-435bd8-kserve-self-signed-certs\""
Nov 03 19:41:19.673807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.673785    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-435bd8\"/\"kube-root-ca.crt\""
Nov 03 19:41:19.674838 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.674820    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-435bd8\"/\"openshift-service-ca.crt\""
Nov 03 19:41:19.677494 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.677469    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"]
Nov 03 19:41:19.719712 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.719676    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41aef14-5753-4120-8136-68d05f9f67f4-tls-certs\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.719866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.719737    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-dshm\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.719928 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.719857    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-home\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.720096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.720076    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-tmp-dir\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.720158 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.720127    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8njt\" (UniqueName: \"kubernetes.io/projected/b41aef14-5753-4120-8136-68d05f9f67f4-kube-api-access-c8njt\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.720275 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.720254    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-model-cache\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.822891 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.822854    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-tmp-dir\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.823110 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.822912    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8njt\" (UniqueName: \"kubernetes.io/projected/b41aef14-5753-4120-8136-68d05f9f67f4-kube-api-access-c8njt\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.823110 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.822985    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-model-cache\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.823242 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.823109    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41aef14-5753-4120-8136-68d05f9f67f4-tls-certs\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.823294 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.823236    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-dshm\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.824128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.824097    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-home\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.824296 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.823916    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-tmp-dir\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.824821 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.824713    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-model-cache\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.825133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.825110    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-home\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.850438 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.850399    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41aef14-5753-4120-8136-68d05f9f67f4-tls-certs\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.862760 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.862722    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-dshm\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.871488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.871458    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8njt\" (UniqueName: \"kubernetes.io/projected/b41aef14-5753-4120-8136-68d05f9f67f4-kube-api-access-c8njt\") pod \"distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") " pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.993467 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.993423    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:19.995904 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:19.995814    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:20.291405 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:20.291380    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"]
Nov 03 19:41:20.292655 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:41:20.292626    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb41aef14_5753_4120_8136_68d05f9f67f4.slice/crio-af56af62c1160d9f96b3dd4ded15e51a6c4318b194188a89fcf498d5eadb7b5f WatchSource:0}: Error finding container af56af62c1160d9f96b3dd4ded15e51a6c4318b194188a89fcf498d5eadb7b5f: Status 404 returned error can't find the container with id af56af62c1160d9f96b3dd4ded15e51a6c4318b194188a89fcf498d5eadb7b5f
Nov 03 19:41:21.211360 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:21.211323    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" event={"ID":"b41aef14-5753-4120-8136-68d05f9f67f4","Type":"ContainerStarted","Data":"438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0"}
Nov 03 19:41:21.211531 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:21.211368    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" event={"ID":"b41aef14-5753-4120-8136-68d05f9f67f4","Type":"ContainerStarted","Data":"af56af62c1160d9f96b3dd4ded15e51a6c4318b194188a89fcf498d5eadb7b5f"}
Nov 03 19:41:21.234559 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:21.234500    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podStartSLOduration=2.234484768 podStartE2EDuration="2.234484768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:21.232388939 +0000 UTC m=+882.140410878" watchObservedRunningTime="2026-11-03 19:41:21.234484768 +0000 UTC m=+882.142506707"
Nov 03 19:41:21.994389 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:21.994358    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:24.554553 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.554512    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"]
Nov 03 19:41:24.558990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.558968    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.574400 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.574374    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"]
Nov 03 19:41:24.615261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.615226    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"]
Nov 03 19:41:24.619581 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.619560    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.634913 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.634889    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"]
Nov 03 19:41:24.684154 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.684113    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"]
Nov 03 19:41:24.689414 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.689386    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.690746 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.690724    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"]
Nov 03 19:41:24.707486 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.707459    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b0156a-0ade-4a21-8835-5b7ec2179330-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.707568 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.707495    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8chrx\" (UniqueName: \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-kube-api-access-8chrx\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.707625 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.707585    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwpvd\" (UniqueName: \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-kube-api-access-mwpvd\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.707662 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.707622    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.707740 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.707723    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.707931 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.707911    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.707979 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.707940    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.707979 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.707957    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.708112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.708097    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.708157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.708118    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.708157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.708134    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.708157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.708150    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.708249 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.708217    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.708284 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.708258    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.708284 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.708280    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.708348 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.708295    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1ac734c-0c72-4335-9d09-5056d87ceb31-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.708348 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.708316    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.708485 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.708464    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.809752 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.809667    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.809752 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.809711    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.809752 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.809739    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.810023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.809769    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.810023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.809821    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.810023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.809846    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.810023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.809869    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.810023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.809894    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.810023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.809930    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.810023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.809957    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0ba4243-e8af-43f4-9296-89099991027e-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.810023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.809985    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.810023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.810014    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.810451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.810073    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.810451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.810101    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1ac734c-0c72-4335-9d09-5056d87ceb31-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.810451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.810119    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.810451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.810126    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.810451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.810360    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.810451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.810428    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.810656 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.810470    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.810701 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.810671    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.810913 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.810889    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.811024 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.810975    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.811147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.811122    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.811147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.811078    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.811272 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.811163    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.811272 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.811230    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b0156a-0ade-4a21-8835-5b7ec2179330-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.811272 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.811262    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jh88\" (UniqueName: \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-kube-api-access-2jh88\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.811419 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.811294    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8chrx\" (UniqueName: \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-kube-api-access-8chrx\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.811419 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.811337    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.811419 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.811382    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwpvd\" (UniqueName: \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-kube-api-access-mwpvd\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.811563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.811415    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1ac734c-0c72-4335-9d09-5056d87ceb31-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.811563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.811429    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.811786 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.811763    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.812216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.812192    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b0156a-0ade-4a21-8835-5b7ec2179330-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.812323 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.812304    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0ba4243-e8af-43f4-9296-89099991027e-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.812395 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.812343    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.812454 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.812397    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.813261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.813235    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.814298 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.814273    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.814694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.814676    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.816887 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.816865    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.824868 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.824841    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.825235 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.825216    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.825955 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.825921    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8chrx\" (UniqueName: \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-kube-api-access-8chrx\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:24.826290 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.826272    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwpvd\" (UniqueName: \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-kube-api-access-mwpvd\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.875522 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.875497    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:24.913652 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.913623    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0ba4243-e8af-43f4-9296-89099991027e-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.913800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.913671    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.913800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.913776    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.913908 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.913817    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0ba4243-e8af-43f4-9296-89099991027e-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.913908 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.913842    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.913908 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.913872    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.914109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.913927    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.914109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.913975    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jh88\" (UniqueName: \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-kube-api-access-2jh88\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.914109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.914012    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.914273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.914156    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-credential-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.914273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.914189    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-data\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.914549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.914518    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-socket\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.914683 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.914651    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-certs\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.914751 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.914732    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0ba4243-e8af-43f4-9296-89099991027e-istiod-ca-cert\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.916256 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.916230    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-envoy\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.916343 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.916288    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0ba4243-e8af-43f4-9296-89099991027e-istio-podinfo\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.929446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.928480    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-istio-token\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.929446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.928611    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jh88\" (UniqueName: \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-kube-api-access-2jh88\") pod \"e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") " pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:24.944090 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:24.942057    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:25.004355 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.004323    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:25.085411 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.085384    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"]
Nov 03 19:41:25.098133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.098110    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.102909 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.102886    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"]
Nov 03 19:41:25.159479 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.159445    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"]
Nov 03 19:41:25.160079 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:41:25.160049    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1ac734c_0c72_4335_9d09_5056d87ceb31.slice/crio-2f5a8ddf2c25ee7c961137b9f8cc032f066d303486a2fb33515301995fde1c1d WatchSource:0}: Error finding container 2f5a8ddf2c25ee7c961137b9f8cc032f066d303486a2fb33515301995fde1c1d: Status 404 returned error can't find the container with id 2f5a8ddf2c25ee7c961137b9f8cc032f066d303486a2fb33515301995fde1c1d
Nov 03 19:41:25.163082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.163030    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.163166 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.163122    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.163209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.163163    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.224092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.224060    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.224217 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.224157    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d276c115-0214-490a-bcd7-206b3beed9d4-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.224281 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.224236    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.224342 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.224299    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d276c115-0214-490a-bcd7-206b3beed9d4-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.224397 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.224376    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.224471 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.224454    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb7lj\" (UniqueName: \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-kube-api-access-hb7lj\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.224576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.224554    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.224682 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.224665    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.224738 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.224712    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.240917 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.240890    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" event={"ID":"c1ac734c-0c72-4335-9d09-5056d87ceb31","Type":"ContainerStarted","Data":"661af47b265a1d0cb28fbdaf3584e7d1a4c30ef1997e65615060913ef6da2c27"}
Nov 03 19:41:25.241083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.240936    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" event={"ID":"c1ac734c-0c72-4335-9d09-5056d87ceb31","Type":"ContainerStarted","Data":"2f5a8ddf2c25ee7c961137b9f8cc032f066d303486a2fb33515301995fde1c1d"}
Nov 03 19:41:25.270110 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.265215    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"]
Nov 03 19:41:25.272360 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.272321    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.282206 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.272495    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.282206 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.272608    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.282206 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.278186    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podStartSLOduration=1.27816201 podStartE2EDuration="1.27816201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:25.273863769 +0000 UTC m=+886.181885709" watchObservedRunningTime="2026-11-03 19:41:25.27816201 +0000 UTC m=+886.186183950"
Nov 03 19:41:25.326089 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.326036    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d276c115-0214-490a-bcd7-206b3beed9d4-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.326213 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.326144    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.326278 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.326243    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb7lj\" (UniqueName: \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-kube-api-access-hb7lj\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.326338 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.326289    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.326424 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.326407    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.326501 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.326459    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.326709 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.326686    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.326837 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.326738    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d276c115-0214-490a-bcd7-206b3beed9d4-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.326837 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.326827    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.326952 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.326897    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.328788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.328325    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d276c115-0214-490a-bcd7-206b3beed9d4-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.330004 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.329976    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d276c115-0214-490a-bcd7-206b3beed9d4-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.336022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.335999    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"]
Nov 03 19:41:25.337412 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.337216    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.337485 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.337459    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.337565 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.337538    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.339259 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.339235    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.340381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.340305    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb7lj\" (UniqueName: \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-kube-api-access-hb7lj\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.341684 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.341654    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.342295 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:41:25.342270    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ba4243_e8af_43f4_9296_89099991027e.slice/crio-2ee2dd3b7f9e5d4fa0af259aacbd13e7ec2b84dd27cc4cdebd1daa472d74f66c WatchSource:0}: Error finding container 2ee2dd3b7f9e5d4fa0af259aacbd13e7ec2b84dd27cc4cdebd1daa472d74f66c: Status 404 returned error can't find the container with id 2ee2dd3b7f9e5d4fa0af259aacbd13e7ec2b84dd27cc4cdebd1daa472d74f66c
Nov 03 19:41:25.344804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.344764    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.344898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.344849    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.344956 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.344893    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.419559 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.419528    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:25.722822 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.722789    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"]
Nov 03 19:41:25.728103 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:41:25.726985    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd276c115_0214_490a_bcd7_206b3beed9d4.slice/crio-5c5d21749c77e960713e747ec6ea1c93b9eb32db6e8fabb518688db8b098a709 WatchSource:0}: Error finding container 5c5d21749c77e960713e747ec6ea1c93b9eb32db6e8fabb518688db8b098a709: Status 404 returned error can't find the container with id 5c5d21749c77e960713e747ec6ea1c93b9eb32db6e8fabb518688db8b098a709
Nov 03 19:41:25.732251 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.732218    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.732333 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.732283    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.732333 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.732311    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:25.876558 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:25.876526    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:26.315443 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.315389    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" event={"ID":"f0ba4243-e8af-43f4-9296-89099991027e","Type":"ContainerStarted","Data":"29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099"}
Nov 03 19:41:26.315615 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.315469    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" event={"ID":"f0ba4243-e8af-43f4-9296-89099991027e","Type":"ContainerStarted","Data":"2ee2dd3b7f9e5d4fa0af259aacbd13e7ec2b84dd27cc4cdebd1daa472d74f66c"}
Nov 03 19:41:26.319447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.319411    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" event={"ID":"96b0156a-0ade-4a21-8835-5b7ec2179330","Type":"ContainerStarted","Data":"897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f"}
Nov 03 19:41:26.319596 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.319475    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" event={"ID":"96b0156a-0ade-4a21-8835-5b7ec2179330","Type":"ContainerStarted","Data":"5d42986614aa730832df330f14cdf0a135a7d310c316651fcfdf01a384c04d21"}
Nov 03 19:41:26.324352 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.324326    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" event={"ID":"d276c115-0214-490a-bcd7-206b3beed9d4","Type":"ContainerStarted","Data":"5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce"}
Nov 03 19:41:26.324684 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.324659    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" event={"ID":"d276c115-0214-490a-bcd7-206b3beed9d4","Type":"ContainerStarted","Data":"5c5d21749c77e960713e747ec6ea1c93b9eb32db6e8fabb518688db8b098a709"}
Nov 03 19:41:26.396064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.395542    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podStartSLOduration=2.395520822 podStartE2EDuration="2.395520822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:26.367854884 +0000 UTC m=+887.275876844" watchObservedRunningTime="2026-11-03 19:41:26.395520822 +0000 UTC m=+887.303542762"
Nov 03 19:41:26.439542 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.439135    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:26.457065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.440516    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podStartSLOduration=1.440497917 podStartE2EDuration="1.440497917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:26.438576491 +0000 UTC m=+887.346598428" watchObservedRunningTime="2026-11-03 19:41:26.440497917 +0000 UTC m=+887.348519856"
Nov 03 19:41:26.475071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.464169    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 03 19:41:26.475071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.464240    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 03 19:41:26.507630 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.507569    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" podStartSLOduration=2.507549406 podStartE2EDuration="2.507549406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:26.505010503 +0000 UTC m=+887.413032442" watchObservedRunningTime="2026-11-03 19:41:26.507549406 +0000 UTC m=+887.415571345"
Nov 03 19:41:27.017066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.968236    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 19:41:27.017066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.968303    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 19:41:27.017066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.968598    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:27.017066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.994182    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 19:41:27.017066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:26.994245    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 19:41:27.028599 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.020003    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:27.028599 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.020128    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 03 19:41:27.028599 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.020169    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 03 19:41:27.421963 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.421859    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 03 19:41:27.427228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.427192    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 03 19:41:27.737179 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.733382    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:27.750469 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.743126    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:27.876509 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.876468    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 19:41:27.876811 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.876781    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 19:41:27.932065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.930765    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:27.949066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.946428    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 19:41:27.949066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:27.946496    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 19:41:28.010157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.009101    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:28.022327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.022297    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 03 19:41:28.049259 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.049219    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 03 19:41:28.054240 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.054115    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:28.301364 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.301241    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:28.423064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.422486    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 03 19:41:28.423064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.422555    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 03 19:41:28.449080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.447142    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:28.461929 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.461893    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:28.594408 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.593207    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:28.757065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.754692    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:28.757065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.755841    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:41:28.905132 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.887463    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 19:41:28.992116 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.991798    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 19:41:28.992312 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.934230    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:41:29.016062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.950463    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 19:41:29.016310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:29.016278    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 19:41:29.016428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:29.009781    5051 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:29.016637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:28.992582    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:41:29.035184 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:29.010107    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 03 19:41:29.598819 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:29.597524    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 03 19:41:29.598819 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:29.230074    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv" podUID="fffba9f9-cad3-4867-9e58-e7272e9b4303" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.132.0.38:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:29.598819 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:29.355702    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:29.602441 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:29.069091    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:41:29.652094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:29.641455    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:41:29.654592 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:29.653916    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:41:29.789066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:29.774622    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:30.004066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:29.994434    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:30.057966 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.057899    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 19:41:30.058479 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.058012    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 19:41:30.062069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.058722    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:41:30.062069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.058783    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:30.062069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.058845    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:41:30.062069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.058906    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:41:30.062069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.058967    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:41:30.062069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.059086    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 03 19:41:30.062069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.059134    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 03 19:41:30.062069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.059229    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 19:41:30.062069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.059259    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 19:41:30.084092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.084056    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 03 19:41:30.090637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.084327    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 03 19:41:30.467127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.446532    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 03 19:41:30.467127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.446778    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 03 19:41:30.599173 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.599131    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:30.631064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.602086    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:30.664336 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.664290    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:30.669238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.669145    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:30.877787 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.877697    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 19:41:30.977296 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.977257    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 19:41:30.978375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.944740    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 19:41:30.978548 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:30.978519    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 19:41:31.016490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.016456    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 03 19:41:31.016741 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.016715    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 03 19:41:31.053491 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.053453    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:41:31.054154 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.054124    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:31.056998 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.055792    5051 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:31.158714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.158606    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/health\": context deadline exceeded"
Nov 03 19:41:31.424503 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.424405    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 03 19:41:31.424888 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.424854    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 03 19:41:31.612221 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.611160    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:41:31.649260 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.647796    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:31.667184 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.666105    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:31.724484 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.724350    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:31.980229 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.976156    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 19:41:31.980229 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.976220    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 19:41:32.008071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:31.997666    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 19:41:32.049310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.037692    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 03 19:41:32.049310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.049263    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 03 19:41:32.054137 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.054098    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:41:32.060832 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.060513    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:32.061054 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.049199    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 19:41:32.127165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.127118    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:32.466866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.451069    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 03 19:41:32.466866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.451131    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 03 19:41:32.674111 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.635588    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:32.674111 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.635877    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:41:32.704425 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.677085    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:32.704425 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.703643    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:32.749183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.731253    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:32.952484 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.952450    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 19:41:32.952783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.952735    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 19:41:32.953392 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.953320    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 19:41:32.953574 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:32.953553    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 19:41:33.075621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.032271    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 03 19:41:33.075621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.032325    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 03 19:41:33.075621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.071122    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:33.075621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.074508    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:41:33.443399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.443303    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 03 19:41:33.452659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.443612    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 03 19:41:33.685246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.683275    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:33.685246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.684993    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:33.728228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.727071    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:33.770064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.747317    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:33.933091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.933027    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 19:41:33.933362 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:33.933328    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 19:41:34.118167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.097858    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 19:41:34.346180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.346003    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 19:41:34.346437 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.115249    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:41:34.347855 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:41:34.101306    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:41:34.348015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.343924    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": context deadline exceeded"
Nov 03 19:41:34.348178 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.156911    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 03 19:41:34.348309 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.348290    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 03 19:41:34.348858 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.115324    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:41:34.521064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.507303    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 03 19:41:34.521064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.507370    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 03 19:41:34.714199 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.714146    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:34.751074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.744278    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:41:34.791119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.744309    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:34.823067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.759112    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:34.957661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.957621    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 19:41:34.957937 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.957903    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 19:41:35.016169 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:34.973501    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 19:41:35.016422 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.016392    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 19:41:35.045034 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.016758    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 03 19:41:35.103071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.076505    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 03 19:41:35.394819 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.355599    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:41:35.395168 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.395132    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:35.421781 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.421751    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 03 19:41:35.422015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.421994    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 03 19:41:35.428015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.427988    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:41:35.535773 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.535342    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:41:35.714384 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.714359    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:41:35.755812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.749198    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:35.805232 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.804269    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:41:35.851296 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.851253    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:35.880524 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.880492    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 19:41:35.880788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.880758    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 19:41:35.888792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.888769    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:41:35.900898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:35.900412    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:41:36.011150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:36.011034    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 03 19:41:36.011390 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:36.011358    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 03 19:41:36.476109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:36.474494    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:36.476109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:36.475838    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:36.479264 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:36.478199    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:41:37.364169 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:37.364121    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:37.366356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:37.366334    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:37.367984 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:37.367966    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:41:37.476064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:37.473134    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:41:37.476064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:37.473197    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:37.951100 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:37.943165    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:41:37.951100 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:37.943229    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:38.197065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:38.196460    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:38.221280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:38.218174    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:38.258801 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:38.255290    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:38.258801 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:38.256549    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:38.258801 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:38.258700    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:41:38.808521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:38.808452    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:38.811031 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:38.809930    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:38.811031 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:38.810762    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:41:39.311315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.311281    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:41:39.318336 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.317939    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:41:39.595647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.595557    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"]
Nov 03 19:41:39.600598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.600574    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.619682 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.619652    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"]
Nov 03 19:41:39.716027 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.715989    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.716027 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.716027    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8bc8\" (UniqueName: \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-kube-api-access-n8bc8\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.716234 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.716173    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.716234 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.716211    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.716307 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.716270    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.716355 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.716333    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.716410 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.716396    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.716448 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.716428    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.716523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.716510    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.817776 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.817748    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.817792    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.817852    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.817881    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.817899    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8bc8\" (UniqueName: \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-kube-api-access-n8bc8\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.817977    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.818013    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.818118    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818626 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.818234    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818626 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.818247    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818626 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.818311    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818626 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.818341    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818626 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.818462    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.818932 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.818861    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.820483 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.820458    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.820710 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.820691    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.830989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.830966    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.831489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.831463    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8bc8\" (UniqueName: \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-kube-api-access-n8bc8\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:39.920834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:39.920777    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:40.007213 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.007185    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:40.025979 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.025749    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:40.214015 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:41:40.213972    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0efb2272_0d6b_4d8e_b14b_4337b37eba1f.slice/crio-6a5dad8a0383dd1fd13608d12424fd41a948db6459ae74eaf749d47b9135de4c WatchSource:0}: Error finding container 6a5dad8a0383dd1fd13608d12424fd41a948db6459ae74eaf749d47b9135de4c: Status 404 returned error can't find the container with id 6a5dad8a0383dd1fd13608d12424fd41a948db6459ae74eaf749d47b9135de4c
Nov 03 19:41:40.214394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.214373    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"]
Nov 03 19:41:40.217079 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.217023    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:40.217162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.217127    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:40.217210 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.217168    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:40.268291 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.268258    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"]
Nov 03 19:41:40.274865 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.274846    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.325952 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.325920    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.326216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.326184    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.326316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.326226    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.326394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.326377    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlmsg\" (UniqueName: \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-kube-api-access-wlmsg\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.326478 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.326445    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.326590 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.326569    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.326651 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.326628    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.326711 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.326661    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.326758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.326711    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.438193 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.438155    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.438388 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.438235    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.438951 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.438878    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.440127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.439275    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.440127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.439343    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.440127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.439418    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.440127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.439456    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.440750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.440356    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.442990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.442955    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"]
Nov 03 19:41:40.443181 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.443164    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.443268 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.443196    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.443350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.443312    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlmsg\" (UniqueName: \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-kube-api-access-wlmsg\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.444354 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.444333    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.444749 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.444720    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.445028 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.445004    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.448226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.448205    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.452275 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.452255    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.468456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.468379    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.485599 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.485569    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlmsg\" (UniqueName: \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-kube-api-access-wlmsg\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.522557 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.522518    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"]
Nov 03 19:41:40.529375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.529351    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.592829 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.592796    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:40.594505 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.593115    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"]
Nov 03 19:41:40.617255 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.617225    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"]
Nov 03 19:41:40.622731 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.622707    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.646381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.646346    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.646520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.646443    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.646520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.646474    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.646650 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.646545    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.646650 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.646586    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.646650 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.646644    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cx9x\" (UniqueName: \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-kube-api-access-7cx9x\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.646811 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.646708    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.647171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.646857    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff19db52-e5c7-4f69-aeb3-876bb9488667-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.653086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.648289    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.661080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.658529    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"]
Nov 03 19:41:40.750182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750095    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9fb4048-87a7-4c19-88f4-c0193b340040-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.750182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750160    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.750404 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750243    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.750404 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750314    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.750404 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750386    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.750563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750424    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.750563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750458    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.750563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750495    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.750714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750560    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.750714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750597    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.750714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750654    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.750714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750701    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.750915 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.750721    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cx9x\" (UniqueName: \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-kube-api-access-7cx9x\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.751251 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.751225    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.751333 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.751298    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.751333 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.751228    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.751434 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.751378    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.751434 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.751389    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff19db52-e5c7-4f69-aeb3-876bb9488667-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.751588 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.751516    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.751667 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.751648    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.751667 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.751648    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.751787 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.751763    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7b2r\" (UniqueName: \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-kube-api-access-l7b2r\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.751967 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.751949    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff19db52-e5c7-4f69-aeb3-876bb9488667-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.753721 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.753695    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.755733 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.755707    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.789741 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.789698    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.790850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.790812    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cx9x\" (UniqueName: \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-kube-api-access-7cx9x\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.865239 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.865190    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:40.866490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.866149    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9fb4048-87a7-4c19-88f4-c0193b340040-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.866490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.866208    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.866490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.866304    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.866490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.866355    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.866490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.866403    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.866490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.866495    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.866925 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.866588    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.866925 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.866610    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.866925 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.866673    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7b2r\" (UniqueName: \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-kube-api-access-l7b2r\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.866925 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.866908    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9fb4048-87a7-4c19-88f4-c0193b340040-istiod-ca-cert\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.867212 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.867189    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-data\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.867272 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.867226    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-credential-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.867654 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.867626    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-socket\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.867745 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.867718    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-certs\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.869306 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.869283    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-podinfo\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.869920 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.869901    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-envoy\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.874518 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.874491    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" event={"ID":"0efb2272-0d6b-4d8e-b14b-4337b37eba1f","Type":"ContainerStarted","Data":"5dfa8d9201e34b7f70a7a97f0e4ef240423af00735575d11aee5a3d143785bb2"}
Nov 03 19:41:40.874614 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.874554    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" event={"ID":"0efb2272-0d6b-4d8e-b14b-4337b37eba1f","Type":"ContainerStarted","Data":"6a5dad8a0383dd1fd13608d12424fd41a948db6459ae74eaf749d47b9135de4c"}
Nov 03 19:41:40.879907 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.879883    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-token\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.880237 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.880219    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7b2r\" (UniqueName: \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-kube-api-access-l7b2r\") pod \"e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") " pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:40.903754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.903687    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" podStartSLOduration=1.903670905 podStartE2EDuration="1.903670905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:40.903313782 +0000 UTC m=+901.811335722" watchObservedRunningTime="2026-11-03 19:41:40.903670905 +0000 UTC m=+901.811692844"
Nov 03 19:41:40.911589 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:41:40.911560    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aea9ee4_a9ae_48f5_aeb8_b6a893b70caa.slice/crio-b7bbcc5d81a896e64c141272dfb294e778bf2f6615a783d20e3173b9a4049260 WatchSource:0}: Error finding container b7bbcc5d81a896e64c141272dfb294e778bf2f6615a783d20e3173b9a4049260: Status 404 returned error can't find the container with id b7bbcc5d81a896e64c141272dfb294e778bf2f6615a783d20e3173b9a4049260
Nov 03 19:41:40.914402 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.912380    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"]
Nov 03 19:41:40.914600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.914568    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:40.914670 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.914634    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:40.914725 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.914686    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:40.921381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.921350    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:40.975330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:40.939302    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:41.183062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.182932    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"]
Nov 03 19:41:41.193069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.193011    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:41.193187 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.193116    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:41.193187 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.193162    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:41.297517 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:41:41.297490    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9fb4048_87a7_4c19_88f4_c0193b340040.slice/crio-74084e28a43348c42a757843538a82cd3e49876c0cc03bc1633f0170f1d4727a WatchSource:0}: Error finding container 74084e28a43348c42a757843538a82cd3e49876c0cc03bc1633f0170f1d4727a: Status 404 returned error can't find the container with id 74084e28a43348c42a757843538a82cd3e49876c0cc03bc1633f0170f1d4727a
Nov 03 19:41:41.316847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.316288    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:41.316847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.316430    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:41.316847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.316519    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:41.316847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.316625    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"]
Nov 03 19:41:41.915371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.915333    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" event={"ID":"e9fb4048-87a7-4c19-88f4-c0193b340040","Type":"ContainerStarted","Data":"174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908"}
Nov 03 19:41:41.915913 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.915444    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" event={"ID":"e9fb4048-87a7-4c19-88f4-c0193b340040","Type":"ContainerStarted","Data":"74084e28a43348c42a757843538a82cd3e49876c0cc03bc1633f0170f1d4727a"}
Nov 03 19:41:41.918105 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.918028    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" event={"ID":"ff19db52-e5c7-4f69-aeb3-876bb9488667","Type":"ContainerStarted","Data":"8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f"}
Nov 03 19:41:41.918207 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.918125    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" event={"ID":"ff19db52-e5c7-4f69-aeb3-876bb9488667","Type":"ContainerStarted","Data":"67620e20f243fa30e4681b83d889f0f0269c7fb2de3bfad22c0bedf0e5340030"}
Nov 03 19:41:41.920593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.920567    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" event={"ID":"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa","Type":"ContainerStarted","Data":"2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f"}
Nov 03 19:41:41.920716 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.920609    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" event={"ID":"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa","Type":"ContainerStarted","Data":"b7bbcc5d81a896e64c141272dfb294e778bf2f6615a783d20e3173b9a4049260"}
Nov 03 19:41:41.922183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.922160    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 19:41:41.922298 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.922204    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 19:41:41.939527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.939505    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:41.957065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:41.955273    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" podStartSLOduration=1.9552528850000002 podStartE2EDuration="1.955252885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:41.939083373 +0000 UTC m=+902.847105310" watchObservedRunningTime="2026-11-03 19:41:41.955252885 +0000 UTC m=+902.863274824"
Nov 03 19:41:42.001109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:42.000321    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" podStartSLOduration=2.000301009 podStartE2EDuration="2.000301009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:41.988536117 +0000 UTC m=+902.896558057" watchObservedRunningTime="2026-11-03 19:41:42.000301009 +0000 UTC m=+902.908322975"
Nov 03 19:41:42.600098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:42.597209    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:42.600098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:42.599598    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 19:41:42.600098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:42.599745    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 19:41:42.890780 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:42.890681    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:42.891183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:42.891159    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 19:41:42.891359 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:42.891334    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 19:41:42.935084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:42.935026    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 19:41:42.935719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:42.935660    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 19:41:42.970245 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:42.970215    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 19:41:42.970520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:42.970490    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 19:41:43.597066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.596527    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 19:41:43.597066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.596593    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 19:41:43.692066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.690146    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:43.715397 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.715344    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:43.776245 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.776183    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:43.796465 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.796418    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:43.903122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.892106    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 19:41:43.903122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.892165    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 19:41:43.943007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.942966    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 19:41:43.943605 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.943566    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 19:41:43.948281 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.948252    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 19:41:43.948481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:43.948458    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 19:41:44.018164 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.018119    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:44.090064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.086685    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:44.090064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.088432    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:44.211119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.201979    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:44.347065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.342125    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:44.610123 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.609495    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 19:41:44.610123 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.609570    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 19:41:44.718194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.718151    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:44.718906 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.718879    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:41:44.761085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.745567    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" podStartSLOduration=4.745547092 podStartE2EDuration="4.745547092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:42.025627396 +0000 UTC m=+902.933649372" watchObservedRunningTime="2026-11-03 19:41:44.745547092 +0000 UTC m=+905.653569031"
Nov 03 19:41:44.893118 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.892119    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:41:44.893118 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.892864    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:41:44.920069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.916214    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:41:44.920069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.917518    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:41:44.927604 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.921782    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 19:41:44.927604 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.921838    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 19:41:44.927604 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.923806    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 19:41:44.927604 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:44.923841    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 19:41:45.265339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.218384    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:45.265339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.218651    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:41:45.265339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.218815    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:41:45.265339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.219000    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:41:45.265339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.219070    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:41:45.265339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.219106    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:45.265339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.235584    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:41:45.265339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.262246    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:45.299094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.269907    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 19:41:45.299094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.269962    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 19:41:45.648386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.648287    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 19:41:45.650114 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.650077    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 19:41:45.708836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.708188    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:45.723270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.722132    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:45.866978 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.866941    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 19:41:45.867234 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.867207    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 19:41:45.892807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.892768    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:45.964357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.948175    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:41:45.965376 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.965261    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 19:41:45.965376 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:45.965313    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 19:41:46.023082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.009007    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 19:41:46.023082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.009089    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 19:41:46.234113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.233995    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:41:46.234725 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.234312    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:46.256065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.234370    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:46.256065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.252205    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:46.621112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.620657    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 19:41:46.621112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.620733    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 19:41:46.712162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.712116    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:46.712980 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.712955    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:41:46.734472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.734432    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:46.868167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.868118    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 19:41:46.868465 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.868419    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 19:41:46.931243 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.931130    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:46.931503 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.931472    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 19:41:46.931616 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.931524    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 19:41:46.952209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.942732    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 19:41:46.952209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.942781    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 19:41:46.975064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:46.956419    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:47.254128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.254079    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:47.268184 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.268140    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:47.270644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.270613    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:41:47.272132 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.272107    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:47.357209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.357179    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:41:47.615120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.611500    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 19:41:47.615394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.615362    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 19:41:47.757062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.756997    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:47.757662 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.757633    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:41:47.876950 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.876836    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 19:41:47.881126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.877205    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 19:41:47.921508 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.921472    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 19:41:47.921787 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.921742    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 19:41:47.963766 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.960206    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:47.963766 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.960591    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:47.997096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.968872    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 19:41:47.997096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:47.968919    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 19:41:48.017076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.016339    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:41:48.088950 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.061541    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:41:48.274074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.273191    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:48.321089 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.317308    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:48.389758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.372021    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:41:48.424359 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.424322    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:48.458071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.439668    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:41:48.520593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.520561    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:41:48.521183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.521159    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:41:48.599991 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.599900    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 19:41:48.600544 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.600489    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 19:41:48.610252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.609147    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:41:48.637203 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.636279    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:41:48.639223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.638693    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:41:48.872637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.872164    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 19:41:48.872637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.872228    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 19:41:48.942229 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.942198    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 19:41:48.965340 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:48.942551    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 19:41:49.406030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:49.405870    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:49.407597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:49.407576    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:49.409121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:49.409059    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:41:49.598417 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:49.598383    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:49.598814 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:49.598795    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:49.599612 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:49.599591    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:41:49.945067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:49.941430    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 19:41:49.945067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:49.941492    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 19:41:50.197418 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:50.197325    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:50.197756 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:50.197732    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:50.199022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:50.198999    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:41:50.944816 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:50.944787    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:50.945305 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:50.945291    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:50.946111 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:50.946093    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:41:55.056078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.056027    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"]
Nov 03 19:41:55.063645 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.063619    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.084792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.084719    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"]
Nov 03 19:41:55.119971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.115218    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"]
Nov 03 19:41:55.158057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.148141    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.158057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.148253    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.158057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.148288    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.158057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.148350    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.158057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.148385    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.158057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.148440    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.158057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.148487    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.158057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.148563    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92w9f\" (UniqueName: \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-kube-api-access-92w9f\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.158057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.148699    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.175537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.175477    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"]
Nov 03 19:41:55.176996 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.175596    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.250926 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.250885    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.251108 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.250996    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.251108 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251029    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.251193 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251125    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.251225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251196    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhzqn\" (UniqueName: \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-kube-api-access-mhzqn\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.251255 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251240    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.251288 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251265    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.251332 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251317    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.251371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251359    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.251403 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251387    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.251457 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251443    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.251521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251489    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.251601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251583    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92w9f\" (UniqueName: \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-kube-api-access-92w9f\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.251635 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251624    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.251705 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251692    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.251767 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251749    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.251833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251820    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.251871 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.251849    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.252813 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.252790    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.255177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.255150    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.255835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.255811    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.256222 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.256201    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.256566 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.256546    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.257208 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.257154    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.267104 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.259786    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.270933 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.270910    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92w9f\" (UniqueName: \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-kube-api-access-92w9f\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.271898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.271879    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.352914 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.352837    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.352914 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.352884    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353164 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.352916    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353164 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.353135    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.353240    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353328 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.353274    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.353335    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhzqn\" (UniqueName: \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-kube-api-access-mhzqn\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353441 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.353404    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353552 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.353533    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353627 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.353588    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-credential-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353679 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.353624    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-data\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.353731    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istiod-ca-cert\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353895 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.353878    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-socket\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.353991 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.353968    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-certs\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.355204 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.355182    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-envoy\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.355633 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.355613    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-podinfo\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.366155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.366123    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-token\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.366342 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.366325    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhzqn\" (UniqueName: \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-kube-api-access-mhzqn\") pod \"e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") " pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.426387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.426354    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:55.511803 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.511774    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:55.716914 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.716887    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"]
Nov 03 19:41:55.721058 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:41:55.720997    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e426eb4_0a92_4fa3_9982_f94cd9a14263.slice/crio-1ae9ac4b138723900490df03d17a7078eb3838fb349c4aa9dc4554e2ea414ea3 WatchSource:0}: Error finding container 1ae9ac4b138723900490df03d17a7078eb3838fb349c4aa9dc4554e2ea414ea3: Status 404 returned error can't find the container with id 1ae9ac4b138723900490df03d17a7078eb3838fb349c4aa9dc4554e2ea414ea3
Nov 03 19:41:55.724763 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.724493    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:55.724763 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.724579    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:55.724940 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.724915    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:55.813073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.813051    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"]
Nov 03 19:41:55.821024 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:41:55.821000    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97eb9373_4612_4ab1_b511_4b8ea18cb78c.slice/crio-bbbbcf3fd86448cd85cb3a53b4221da618c1266964c003ca66161f54bf58d207 WatchSource:0}: Error finding container bbbbcf3fd86448cd85cb3a53b4221da618c1266964c003ca66161f54bf58d207: Status 404 returned error can't find the container with id bbbbcf3fd86448cd85cb3a53b4221da618c1266964c003ca66161f54bf58d207
Nov 03 19:41:55.823621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.823590    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:55.823708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.823655    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:55.823708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:55.823683    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:41:56.724090 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:56.724051    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" event={"ID":"97eb9373-4612-4ab1-b511-4b8ea18cb78c","Type":"ContainerStarted","Data":"16f478f6fb9ed866513d786dfc713cba63b996b50a99f414dd7e70c9db0a5122"}
Nov 03 19:41:56.724620 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:56.724113    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" event={"ID":"97eb9373-4612-4ab1-b511-4b8ea18cb78c","Type":"ContainerStarted","Data":"bbbbcf3fd86448cd85cb3a53b4221da618c1266964c003ca66161f54bf58d207"}
Nov 03 19:41:56.725942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:56.725913    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" event={"ID":"6e426eb4-0a92-4fa3-9982-f94cd9a14263","Type":"ContainerStarted","Data":"36a498222fbedac613e1f161279f33dd21edd6df044cfdca5b3a38ed70d4ea62"}
Nov 03 19:41:56.726096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:56.725957    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" event={"ID":"6e426eb4-0a92-4fa3-9982-f94cd9a14263","Type":"ContainerStarted","Data":"1ae9ac4b138723900490df03d17a7078eb3838fb349c4aa9dc4554e2ea414ea3"}
Nov 03 19:41:56.768081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:56.767995    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podStartSLOduration=1.767975246 podStartE2EDuration="1.767975246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:56.767942605 +0000 UTC m=+917.675964545" watchObservedRunningTime="2026-11-03 19:41:56.767975246 +0000 UTC m=+917.675997185"
Nov 03 19:41:56.805538 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:56.805480    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podStartSLOduration=1.8054645919999999 podStartE2EDuration="1.805464592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:41:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:41:56.796727183 +0000 UTC m=+917.704749149" watchObservedRunningTime="2026-11-03 19:41:56.805464592 +0000 UTC m=+917.713486532"
Nov 03 19:41:57.427640 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:57.427607    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:41:57.428879 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:57.428726    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 03 19:41:57.428879 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:57.428773    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 03 19:41:57.512559 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:57.512525    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:41:57.513846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:57.513825    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 03 19:41:57.513921 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:57.513874    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 03 19:41:58.108589 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:58.108556    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:41:58.426867 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:58.426774    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 03 19:41:58.427147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:58.427109    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 03 19:41:58.512551 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:58.512516    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 03 19:41:58.512714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:58.512565    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 03 19:41:59.433833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:59.433797    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 03 19:41:59.434443 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:59.434412    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 03 19:41:59.514520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:59.514480    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 03 19:41:59.514736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:41:59.514570    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 03 19:42:00.428990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:00.428957    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 03 19:42:00.429271 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:00.429243    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 03 19:42:00.515112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:00.513477    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 03 19:42:00.515112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:00.513546    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 03 19:42:00.766065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:00.765134    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:42:00.956627 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:00.954135    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:42:01.000190 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.000134    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:01.080032 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.079908    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:42:01.174066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.173151    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:01.240332 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.240252    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:01.262172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.262127    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:42:01.277177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.277134    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:01.319176 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.319122    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:01.438677 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.429986    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 03 19:42:01.463067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.438912    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 03 19:42:01.554222 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.519509    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 03 19:42:01.562062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.554794    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 03 19:42:01.805277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.805232    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:42:01.839535 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.839504    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:42:01.977311 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:01.977254    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:42:02.005225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.001894    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:42:02.005225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.002570    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:42:02.106139 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.093538    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:42:02.138081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.106850    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:42:02.138081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.136665    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:02.353063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.352795    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:42:02.374124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.354911    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:42:02.374124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.370527    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:42:02.374124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.370608    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:42:02.397214 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.397168    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:02.409066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.406624    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:42:02.441115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.426695    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:02.441115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.430945    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:42:02.468116 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.456598    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 03 19:42:02.468116 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.456670    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 03 19:42:02.531066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.530680    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 03 19:42:02.605066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.530759    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 03 19:42:02.639957 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:02.639856    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:03.121069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.120171    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:42:03.121404 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.121370    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:03.434566 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.217755    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:42:03.476078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.396184    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:03.476735 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.404992    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:42:03.477266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.410099    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:42:03.513359 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.513305    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:03.602714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.602669    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:42:03.705964 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.705874    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:42:03.788065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.732386    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 03 19:42:03.788362 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.788327    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 03 19:42:03.813062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:03.762896    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 03 19:42:04.419194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.419144    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 03 19:42:04.419885 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.350483    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:42:04.423152 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.423118    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:04.490068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.451592    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": context deadline exceeded"
Nov 03 19:42:04.490068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.475909    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:42:04.490068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.476000    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:04.520624 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.520580    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 03 19:42:04.647236 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.647189    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 03 19:42:04.647517 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.561196    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv" podUID="fffba9f9-cad3-4867-9e58-e7272e9b4303" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.132.0.38:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:04.647636 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.562086    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:42:04.652064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.560255    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:42:04.730126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.679216    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:42:04.730126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.679279    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:04.730126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.563514    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:42:04.730126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.563486    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:42:04.730126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.563535    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:42:04.730126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.632157    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:04.730126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.583087    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 03 19:42:04.730126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.683965    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 03 19:42:04.730126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.563412    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:04.730126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:04.563463    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:42:05.006354 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.006254    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:42:05.006617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.006583    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:05.017695 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:42:05.014534    5051 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.252s"
Nov 03 19:42:05.261685 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:42:05.261572    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:42:05.474067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.454187    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:42:05.474438 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.474390    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:05.474635 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.473500    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 03 19:42:05.474780 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.474759    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 03 19:42:05.475059 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.475022    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:42:05.475251 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.475226    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:05.528267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.528178    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 03 19:42:05.528507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.528478    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 03 19:42:05.670067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.666173    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:42:05.670067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.667544    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:05.693066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.689057    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:05.717073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.708421    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:05.717073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.709643    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:05.717073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.715581    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:42:05.717073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.715790    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:05.735363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.734194    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:42:05.918575 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.918447    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:42:05.918893 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:05.918860    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:06.428892 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.428840    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 03 19:42:06.429822 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.429211    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 03 19:42:06.513329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.512894    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 03 19:42:06.513329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.512954    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 03 19:42:06.656067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.612150    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:42:06.700765 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.677194    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:06.700765 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.680664    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:06.700765 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.698407    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:06.702876 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.701702    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:42:06.711364 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.711333    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:06.716513 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.716484    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:06.765651 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.765471    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:06.782071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.766655    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:42:06.798642 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.785090    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:42:06.819539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.819497    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:42:06.820448 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.820422    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:42:06.830208 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.829483    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:42:06.832094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:06.832075    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:42:07.427175 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:07.427135    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 03 19:42:07.427343 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:07.427205    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 03 19:42:07.512618 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:07.512581    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 03 19:42:07.512802 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:07.512636    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 03 19:42:08.522136 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:08.521572    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:42:08.522136 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:08.522108    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:42:08.523207 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:08.523184    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:42:09.468671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:09.468634    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:42:09.469189 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:09.469156    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:09.760453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:09.760352    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:42:09.762083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:09.762024    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:42:09.803664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:09.803635    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:42:24.689277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.688888    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"]
Nov 03 19:42:24.695862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.695835    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.713113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.713082    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"]
Nov 03 19:42:24.827187 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.827152    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.827369 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.827277    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef3fd8a9-7545-4236-86f0-f164991045df-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.827434 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.827402    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.827518 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.827494    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.827598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.827574    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.827656 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.827641    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.828100 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.827691    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef3fd8a9-7545-4236-86f0-f164991045df-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.828100 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.827779    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrvz7\" (UniqueName: \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-kube-api-access-lrvz7\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.828100 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.827816    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.934200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.933280    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrvz7\" (UniqueName: \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-kube-api-access-lrvz7\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.934200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.933368    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.934200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.933502    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.934200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.933593    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef3fd8a9-7545-4236-86f0-f164991045df-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.934200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.933682    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.934200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.933726    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.934200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.933762    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.934200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.933841    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.934200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.933954    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef3fd8a9-7545-4236-86f0-f164991045df-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.934993 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.934968    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef3fd8a9-7545-4236-86f0-f164991045df-istiod-ca-cert\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.935672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.935646    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-certs\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.940436 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.940366    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-data\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.941030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.940995    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.941193 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.941132    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-credential-socket\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.942884 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.942863    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-envoy\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.943536 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.943517    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef3fd8a9-7545-4236-86f0-f164991045df-istio-podinfo\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.951563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.951533    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrvz7\" (UniqueName: \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-kube-api-access-lrvz7\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:24.955849 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:24.955821    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-istio-token\") pod \"e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") " pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:25.018474 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.018444    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:25.308108 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:42:25.308078    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef3fd8a9_7545_4236_86f0_f164991045df.slice/crio-3ca78c2ccdd411943b50104a0d8f66f56114c1731ffbbfd9ce8511010d7a60f2 WatchSource:0}: Error finding container 3ca78c2ccdd411943b50104a0d8f66f56114c1731ffbbfd9ce8511010d7a60f2: Status 404 returned error can't find the container with id 3ca78c2ccdd411943b50104a0d8f66f56114c1731ffbbfd9ce8511010d7a60f2
Nov 03 19:42:25.309054 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.309018    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"]
Nov 03 19:42:25.310905 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.310869    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:25.310981 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.310951    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:25.311017 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.310991    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:25.443173 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.443140    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"]
Nov 03 19:42:25.450575 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.449639    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.452079 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.449539    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"]
Nov 03 19:42:25.546362 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.546334    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h25z8\" (UniqueName: \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-kube-api-access-h25z8\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.546560 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.546409    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.546560 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.546502    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.546560 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.546528    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.546697 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.546641    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.546750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.546705    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.546868 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.546840    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.547089 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.547036    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.547176 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.547159    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.547532 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.547512    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" event={"ID":"ef3fd8a9-7545-4236-86f0-f164991045df","Type":"ContainerStarted","Data":"676ec120c450ea6af22731af7fe1bc5486c1e7094851152feee20da2e30a282c"}
Nov 03 19:42:25.547592 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.547556    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" event={"ID":"ef3fd8a9-7545-4236-86f0-f164991045df","Type":"ContainerStarted","Data":"3ca78c2ccdd411943b50104a0d8f66f56114c1731ffbbfd9ce8511010d7a60f2"}
Nov 03 19:42:25.572703 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.572613    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podStartSLOduration=1.572595897 podStartE2EDuration="1.572595897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:42:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:42:25.570063124 +0000 UTC m=+946.478085064" watchObservedRunningTime="2026-11-03 19:42:25.572595897 +0000 UTC m=+946.480617836"
Nov 03 19:42:25.648794 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.648764    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.648964 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.648853    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.649163 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.649134    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h25z8\" (UniqueName: \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-kube-api-access-h25z8\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.649326 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.649210    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.649326 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.649267    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.649326 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.649292    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.649510 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.649321    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-certs\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.649510 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.649326    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.649626 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.649541    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.649667 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.649637    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.649706 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.649662    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-data\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.650005 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.649978    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istiod-ca-cert\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.650185 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.650166    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.650294 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.650276    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-credential-socket\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.652057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.652026    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-podinfo\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.652160 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.652030    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-envoy\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.661700 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.661677    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h25z8\" (UniqueName: \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-kube-api-access-h25z8\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.661872 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.661854    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-token\") pod \"e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") " pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:25.777413 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:25.777023    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:26.018890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:26.018854    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:26.020387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:26.020360    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 03 19:42:26.020512 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:26.020411    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 03 19:42:26.102470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:26.098418    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"]
Nov 03 19:42:26.102679 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:42:26.102505    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2d6acc6_3543_4cd5_ac03_38a9b412f25b.slice/crio-d1f0ca1b09bcc4b709dc8e2f7458c13dc2653eea28e816f7c55fbc7a9a5d08bc WatchSource:0}: Error finding container d1f0ca1b09bcc4b709dc8e2f7458c13dc2653eea28e816f7c55fbc7a9a5d08bc: Status 404 returned error can't find the container with id d1f0ca1b09bcc4b709dc8e2f7458c13dc2653eea28e816f7c55fbc7a9a5d08bc
Nov 03 19:42:26.104845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:26.104813    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:26.105000 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:26.104879    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:26.105000 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:26.104909    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:26.577511 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:26.577471    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" event={"ID":"c2d6acc6-3543-4cd5-ac03-38a9b412f25b","Type":"ContainerStarted","Data":"42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30"}
Nov 03 19:42:26.578187 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:26.578157    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" event={"ID":"c2d6acc6-3543-4cd5-ac03-38a9b412f25b","Type":"ContainerStarted","Data":"d1f0ca1b09bcc4b709dc8e2f7458c13dc2653eea28e816f7c55fbc7a9a5d08bc"}
Nov 03 19:42:26.606635 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:26.606567    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podStartSLOduration=1.6065284119999999 podStartE2EDuration="1.606528412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:42:26.600636705 +0000 UTC m=+947.508658657" watchObservedRunningTime="2026-11-03 19:42:26.606528412 +0000 UTC m=+947.514550352"
Nov 03 19:42:26.778128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:26.778097    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:27.019008 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:27.018970    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 03 19:42:27.019228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:27.019026    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 03 19:42:27.793068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:27.792169    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 03 19:42:27.793068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:27.792244    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 03 19:42:28.019251 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:28.019214    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 03 19:42:28.019452 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:28.019274    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 03 19:42:28.788069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:28.787958    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 03 19:42:28.788069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:28.788031    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 03 19:42:29.019481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:29.019434    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 03 19:42:29.019878 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:29.019504    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 03 19:42:29.790869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:29.780403    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 03 19:42:29.790869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:29.780462    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 03 19:42:30.038132 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:30.031479    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 03 19:42:30.038132 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:30.031563    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 03 19:42:30.650671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:30.650628    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:42:30.691182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:30.691135    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:30.744069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:30.721550    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:42:30.744069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:30.733569    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:30.778848 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:30.778799    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 03 19:42:30.796064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:30.779112    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 03 19:42:30.824189 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:30.824141    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:31.002193 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.002137    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:31.034208 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.034034    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 03 19:42:31.034482 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.034452    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 03 19:42:31.122454 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.122400    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:31.156850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.156807    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/health\": context deadline exceeded"
Nov 03 19:42:31.666110 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.663943    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:31.666110 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.665345    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:42:31.760659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.703932    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:31.791084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.768251    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:42:31.794555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.704087    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:31.799204 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.740341    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:31.799204 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.740282    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:42:31.824261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.813642    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:42:31.824261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.813704    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:42:31.826135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.824960    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:31.851284 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.827396    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:42:31.851284 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.828262    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:31.932119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.926364    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 03 19:42:31.942632 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:31.942589    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 03 19:42:32.054066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.027150    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 03 19:42:32.054066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.027212    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 03 19:42:32.224131 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.121796    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:32.224131 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.139557    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:32.224131 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.139908    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:42:32.694137 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.693897    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:32.745065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.727140    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:42:32.774896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.774853    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:32.857060 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.812300    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:32.857060 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.814280    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:32.857060 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.818007    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:32.857060 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.818962    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:42:32.895063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.876177    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:42:32.895063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.879155    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:42:32.895063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.879459    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:42:32.937072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.929005    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 03 19:42:32.937072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:32.929083    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 03 19:42:33.038064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:33.030976    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 03 19:42:33.063447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:33.043281    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 03 19:42:33.165498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:33.165341    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:33.787066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:33.771248    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:33.826067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:33.792140    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:33.826067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:33.793309    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 03 19:42:33.826067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:33.793353    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 03 19:42:33.826067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:33.821706    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:42:33.826067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:33.822100    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:42:33.826067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:33.823255    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:42:33.921078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:33.911917    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:42:34.036021 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.035985    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 03 19:42:34.036229 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.036056    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 03 19:42:34.123486 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.123374    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:42:34.150712 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.150681    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:42:34.193833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.193272    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:42:34.235843 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.235800    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv" podUID="fffba9f9-cad3-4867-9e58-e7272e9b4303" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.132.0.38:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:42:34.294069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.275267    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:42:34.296103 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.296078    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:42:34.297267 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:42:34.297231    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:42:34.314753 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.312399    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:42:34.328904 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.328769    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:42:34.332354 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.332330    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:42:34.333393 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.333373    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:42:34.386266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.386175    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:42:34.784240 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.783696    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 03 19:42:34.784240 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:34.783761    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 03 19:42:35.019427 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:35.019390    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 03 19:42:35.019833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:35.019467    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 03 19:42:35.778235 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:35.778205    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Nov 03 19:42:35.778400 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:35.778260    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Nov 03 19:42:36.024335 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:36.024301    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:36.024857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:36.024802    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:36.025561 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:36.025541    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:42:36.782852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:36.782820    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:36.783263 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:36.783238    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:36.784075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:36.784024    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:42:39.953547 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:39.953514    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"]
Nov 03 19:42:39.959478 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:39.959459    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:39.975449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:39.975423    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"]
Nov 03 19:42:40.082849 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.082816    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-data\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.083031 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.082858    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.083031 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.082932    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istiod-ca-cert\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.083031 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.082994    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-credential-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.083227 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.083202    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-certs\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.083290 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.083238    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-envoy\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.083349 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.083322    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt8fw\" (UniqueName: \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-kube-api-access-pt8fw\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.083403 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.083373    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-token\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.083457 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.083438    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-podinfo\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.184834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.184795    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-certs\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.184834 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.184838    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-envoy\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.185121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.184879    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt8fw\" (UniqueName: \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-kube-api-access-pt8fw\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.185121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.184921    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-token\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.185121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.184964    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-podinfo\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.185293 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.185212    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-data\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.185293 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.185223    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-certs\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.185293 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.185246    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.185293 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.185279    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istiod-ca-cert\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.185494 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.185341    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-credential-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.185542 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.185482    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-data\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.185612 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.185593    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.185789 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.185761    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-credential-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.186160 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.186133    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istiod-ca-cert\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.187401 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.187381    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-envoy\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.187558 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.187538    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-podinfo\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.197901 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.197881    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-token\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.199775 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.199757    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt8fw\" (UniqueName: \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-kube-api-access-pt8fw\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.278267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.278244    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:40.565025 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:42:40.564991    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b3a3536_bf26_42f1_8cb7_e3bb19602c16.slice/crio-33dfd8b2bc6c1c90a6ff2b4e7bb5531f9830f638d73a34587d9206a3f3c9ba67 WatchSource:0}: Error finding container 33dfd8b2bc6c1c90a6ff2b4e7bb5531f9830f638d73a34587d9206a3f3c9ba67: Status 404 returned error can't find the container with id 33dfd8b2bc6c1c90a6ff2b4e7bb5531f9830f638d73a34587d9206a3f3c9ba67
Nov 03 19:42:40.565195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.565091    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"]
Nov 03 19:42:40.567914 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.567877    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:40.568018 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.567961    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:40.568018 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:40.568003    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:41.382908 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:41.382865    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" event={"ID":"6b3a3536-bf26-42f1-8cb7-e3bb19602c16","Type":"ContainerStarted","Data":"845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d"}
Nov 03 19:42:41.383305 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:41.382939    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" event={"ID":"6b3a3536-bf26-42f1-8cb7-e3bb19602c16","Type":"ContainerStarted","Data":"33dfd8b2bc6c1c90a6ff2b4e7bb5531f9830f638d73a34587d9206a3f3c9ba67"}
Nov 03 19:42:41.433706 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:41.433661    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" podStartSLOduration=2.43364575 podStartE2EDuration="2.43364575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:42:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:42:41.432793237 +0000 UTC m=+962.340815195" watchObservedRunningTime="2026-11-03 19:42:41.43364575 +0000 UTC m=+962.341667689"
Nov 03 19:42:42.278515 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:42.278479    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:42.280595 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:42.280566    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 19:42:42.280720 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:42.280630    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" podUID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 19:42:43.278705 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:43.278670    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 19:42:43.279139 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:43.278735    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" podUID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 19:42:44.279051 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:44.279004    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 19:42:44.279471 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:44.279083    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" podUID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 19:42:45.279426 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:45.279392    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 19:42:45.279884 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:45.279455    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" podUID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 19:42:46.278622 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:46.278587    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 19:42:46.278838 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:46.278652    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" podUID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 19:42:47.279655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:47.279613    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 19:42:47.280090 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:47.279675    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" podUID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 19:42:48.285463 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:48.285428    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:48.286510 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:48.286484    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:48.287434 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:48.287412    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:42:55.298653 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.298618    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"]
Nov 03 19:42:55.304869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.304846    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.320242 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.320219    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"]
Nov 03 19:42:55.325708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.325684    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.326658 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.326634    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"]
Nov 03 19:42:55.339771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.339741    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"]
Nov 03 19:42:55.404885 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.404853    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.405059 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.404976    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.405109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.405072    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.405186 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.405173    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.405233 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.405205    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.405352 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.405323    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hpf7\" (UniqueName: \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-kube-api-access-8hpf7\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.405531 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.405367    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.405531 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.405412    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.405531 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.405504    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.506669 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.506635    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.506833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.506694    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13925e4e-1b1f-4306-af1d-418c993e3fb8-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.506833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.506731    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.506833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.506763    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hpf7\" (UniqueName: \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-kube-api-access-8hpf7\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.506963 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.506874    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.506963 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.506939    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.507074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507014    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.507130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507078    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.507130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507080    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.507246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507187    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp7hn\" (UniqueName: \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-kube-api-access-bp7hn\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.507246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507215    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.507246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507242    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.507431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507261    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.507431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507382    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.507539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507470    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.507599 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507583    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.507651 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507629    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.507701 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507658    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.507757 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507708    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.507799 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507783    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.507841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507797    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.507878 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507848    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.507917 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.507891    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.509330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.509312    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.510073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.510056    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.519661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.519637    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.519912 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.519894    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hpf7\" (UniqueName: \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-kube-api-access-8hpf7\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.609645 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.609566    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.609782 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.609707    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp7hn\" (UniqueName: \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-kube-api-access-bp7hn\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.609782 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.609748    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.609917 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.609888    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.609969 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.609945    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.610029 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.609987    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.610174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.610028    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.610174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.610104    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13925e4e-1b1f-4306-af1d-418c993e3fb8-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.610174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.610147    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.610327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.610238    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-data\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.610327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.610295    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-certs\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.610504 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.610479    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.610589 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.610567    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-credential-socket\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.610895 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.610871    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13925e4e-1b1f-4306-af1d-418c993e3fb8-istiod-ca-cert\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.612176 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.612153    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-envoy\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.612285 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.612233    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-podinfo\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.625099 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.625037    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:55.625742 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.625721    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-token\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.626231 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.626212    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp7hn\" (UniqueName: \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-kube-api-access-bp7hn\") pod \"e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") " pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.658177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.647921    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:55.949566 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:42:55.949533    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee9dca8_9896_4cda_b5e4_6c63aa7a268f.slice/crio-4edced7c46c580e292927878ba327485ed34a6879939ac865f03a1cddda7de69 WatchSource:0}: Error finding container 4edced7c46c580e292927878ba327485ed34a6879939ac865f03a1cddda7de69: Status 404 returned error can't find the container with id 4edced7c46c580e292927878ba327485ed34a6879939ac865f03a1cddda7de69
Nov 03 19:42:55.951537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.951516    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"]
Nov 03 19:42:55.952486 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.952453    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:55.952565 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.952518    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:55.952565 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.952547    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:55.998183 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:42:55.998154    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13925e4e_1b1f_4306_af1d_418c993e3fb8.slice/crio-3e94e0bc7286ec430265463b7d69b38da5649f7aec88581bcff2cc76381b5293 WatchSource:0}: Error finding container 3e94e0bc7286ec430265463b7d69b38da5649f7aec88581bcff2cc76381b5293: Status 404 returned error can't find the container with id 3e94e0bc7286ec430265463b7d69b38da5649f7aec88581bcff2cc76381b5293
Nov 03 19:42:55.999603 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:55.999579    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"]
Nov 03 19:42:56.001033 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:56.001006    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:56.001139 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:56.001079    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:56.001139 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:56.001109    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:42:56.565676 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:56.565636    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" event={"ID":"13925e4e-1b1f-4306-af1d-418c993e3fb8","Type":"ContainerStarted","Data":"0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4"}
Nov 03 19:42:56.566094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:56.565700    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" event={"ID":"13925e4e-1b1f-4306-af1d-418c993e3fb8","Type":"ContainerStarted","Data":"3e94e0bc7286ec430265463b7d69b38da5649f7aec88581bcff2cc76381b5293"}
Nov 03 19:42:56.567617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:56.567589    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" event={"ID":"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f","Type":"ContainerStarted","Data":"0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8"}
Nov 03 19:42:56.567733 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:56.567641    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" event={"ID":"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f","Type":"ContainerStarted","Data":"4edced7c46c580e292927878ba327485ed34a6879939ac865f03a1cddda7de69"}
Nov 03 19:42:56.592369 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:56.592320    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podStartSLOduration=1.592304106 podStartE2EDuration="1.592304106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:42:56.588928171 +0000 UTC m=+977.496950113" watchObservedRunningTime="2026-11-03 19:42:56.592304106 +0000 UTC m=+977.500326046"
Nov 03 19:42:56.617762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:56.617700    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podStartSLOduration=1.617686915 podStartE2EDuration="1.617686915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:42:56.615704541 +0000 UTC m=+977.523726503" watchObservedRunningTime="2026-11-03 19:42:56.617686915 +0000 UTC m=+977.525708866"
Nov 03 19:42:56.625769 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:56.625735    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:42:56.648668 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:56.648641    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:42:57.640013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:57.636211    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:42:57.640013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:57.636293    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:42:57.681064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:57.665218    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:42:57.681064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:57.665306    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:42:58.627814 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:58.625875    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:42:58.627814 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:58.627176    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:42:58.680672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:58.680416    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:42:58.680672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:58.680475    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:42:59.495286 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:59.495226    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:42:59.627886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:59.626781    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:42:59.627886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:59.626833    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:42:59.671617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:59.671583    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:42:59.671895 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:42:59.671864    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:00.627069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:00.626597    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:00.627069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:00.626660    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:00.657065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:00.656145    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:00.657065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:00.656219    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:01.362068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:01.331973    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:01.691630 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:01.691538    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:01.692232 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:01.692197    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:01.717403 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:01.717340    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:01.742200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:01.742152    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:01.750611 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:01.750570    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:01.853083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:01.851777    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:01.853083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:01.852906    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:43:01.853358 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:01.853172    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:43:01.901065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:01.894211    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:02.130066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.129296    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:02.211065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.210150    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:02.335429 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.334147    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:02.337158 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.335873    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:43:02.670068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.666161    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:02.725307 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.725264    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:02.725870 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.725836    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:02.726465 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.726440    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:02.748336 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.748299    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:02.773067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.761126    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:43:02.773067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.761611    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:43:02.856196 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.856135    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:02.891073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.856779    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:02.891444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.871254    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:43:02.895062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.891642    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:43:03.458167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.871663    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:03.458167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.902390    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:43:03.458167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:02.979285    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:03.458167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.218376    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:03.458167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.355474    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:03.629430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.628985    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:03.629430 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.629064    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:03.694067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.692217    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:03.694067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.692279    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:03.747311 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:43:03.747271    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:43:03.786183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.786135    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:03.897921 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.879098    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:03.897921 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.894303    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:03.930080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.921770    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:43:03.934260 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.934226    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:43:03.934513 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:03.934494    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:43:04.328287 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.328220    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:04.442492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.438154    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:04.442492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.438226    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:04.443343 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.442857    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:04.443343 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.443279    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:04.444332 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.443827    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:43:04.444332 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.444100    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:04.459067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.444599    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:43:04.497215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.497163    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:04.610343 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.610222    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:04.615194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.615133    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:04.698071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.697882    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:04.698071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.697943    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:04.774082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.756193    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:04.774082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.756261    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:04.774082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.756985    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:04.774082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.757027    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:04.815872 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.815833    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:43:04.879474 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.879376    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:43:04.909212 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:04.909158    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:05.146250 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.146147    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:05.146524 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.146490    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:05.363069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.354703    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:05.363069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.354822    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:05.573899 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.573854    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:43:05.575252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.575218    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:43:05.584063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.575934    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:43:05.609676 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.579242    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:05.610978 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.595149    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:05.685350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.685315    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:05.685718 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.685687    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:05.716682 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.716644    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:05.716988 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.716953    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:05.757216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.757175    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:05.757512 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.757481    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:05.817199 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.817159    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:05.894297 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.894093    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:43:05.912067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:05.910575    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:43:06.399235 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.399196    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:06.434067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.411133    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:06.594210 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.594164    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:43:06.595611 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.595567    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:06.611226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.611174    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:06.615199 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.615161    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:06.643227 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.643186    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:06.643506 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.643477    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:06.649072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.645188    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:06.654112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.650196    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:06.654310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.654280    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:06.832095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.828613    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:06.907221 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.907114    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:06.912153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.912113    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:06.992226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.992186    5051 patch_prober.go:29] interesting pod/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:06.992488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:06.992457    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:07.111069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.109154    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:07.111069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.109225    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:07.612090 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.612019    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:43:07.640516 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.640477    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:07.645155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.645117    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:07.681066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.641647    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:43:07.685066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.641679    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:43:07.689063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.641612    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:07.731495 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.649338    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:07.737210 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.736961    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:07.737210 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.737139    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:07.739900 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.739820    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:07.829275 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.829218    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:07.837166 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.837130    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:07.837723 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.837693    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:07.912126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.909553    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:07.930413 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:07.930375    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:43:08.615190 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.615139    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:08.628513 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.628473    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:08.628790 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.628760    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:08.649984 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.649941    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:08.650350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.650328    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:08.650498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.650476    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:08.684730 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.684686    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:08.685911 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.685876    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:43:08.742180 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.742130    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:43:08.742811 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.742777    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:43:08.836349 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.836302    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:08.923282 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.923175    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:08.932905 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:08.932857    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:09.033135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.033086    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:09.191290 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.191171    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv" podUID="fffba9f9-cad3-4867-9e58-e7272e9b4303" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.132.0.38:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:09.317239 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.317191    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:09.317561 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.317519    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:09.624096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.621236    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:09.624096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.621808    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:09.624096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.621854    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:09.649064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.642121    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:09.649064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.642247    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:09.649427 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.649400    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:09.649571 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.649550    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:09.694195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.694143    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:09.736064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.694459    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:09.854318 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.854273    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:43:10.091202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.863113    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:10.091202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.863402    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:43:10.091202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.942627    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:43:10.091202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:09.943732    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:43:10.658101 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:10.658067    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:43:10.684323 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:10.684282    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:43:10.831073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:10.729862    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:10.831073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:10.816207    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:10.831073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:10.732980    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:43:10.831073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:10.730091    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:10.831073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:10.787080    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:43:11.087842 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.085704    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:11.087842 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.085783    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:11.087842 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.087242    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": context deadline exceeded"
Nov 03 19:43:11.087842 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.087299    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:43:11.088338 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.088314    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:43:11.088929 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.088904    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:11.089563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.089542    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:43:11.095058 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.091998    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:43:11.140316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.140262    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n" podUID="c8f9579a-928c-41e0-a4ad-3438618bae19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:11.192636 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.191032    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:11.194609 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.194584    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:43:11.201825 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.201802    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:43:11.203808 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.203788    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:43:11.205225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.205206    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:43:11.211321 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.211305    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:43:11.214390 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.214373    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:43:11.214591 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.214564    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:43:11.218579 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.218560    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:43:11.600948 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.600912    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7098-50265c"]
Nov 03 19:43:11.607339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.607309    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7098-50265c"
Nov 03 19:43:11.621166 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.621134    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7098-50265c"]
Nov 03 19:43:11.634719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.632770    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 19:43:11.634719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.632825    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 19:43:11.634719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.633510    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-hjqcx\""
Nov 03 19:43:11.669917 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.669873    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 19:43:11.670204 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.670156    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 19:43:11.708568 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.708529    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv7gw\" (UniqueName: \"kubernetes.io/projected/0c3e7108-2bb3-4619-9154-9675be4b82b0-kube-api-access-pv7gw\") pod \"test-curl-7098-50265c\" (UID: \"0c3e7108-2bb3-4619-9154-9675be4b82b0\") " pod="opendatahub/test-curl-7098-50265c"
Nov 03 19:43:11.814292 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.814252    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv7gw\" (UniqueName: \"kubernetes.io/projected/0c3e7108-2bb3-4619-9154-9675be4b82b0-kube-api-access-pv7gw\") pod \"test-curl-7098-50265c\" (UID: \"0c3e7108-2bb3-4619-9154-9675be4b82b0\") " pod="opendatahub/test-curl-7098-50265c"
Nov 03 19:43:11.828393 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.828366    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv7gw\" (UniqueName: \"kubernetes.io/projected/0c3e7108-2bb3-4619-9154-9675be4b82b0-kube-api-access-pv7gw\") pod \"test-curl-7098-50265c\" (UID: \"0c3e7108-2bb3-4619-9154-9675be4b82b0\") " pod="opendatahub/test-curl-7098-50265c"
Nov 03 19:43:11.992393 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:11.992351    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7098-50265c"
Nov 03 19:43:12.358156 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:12.357996    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7098-50265c"]
Nov 03 19:43:12.370651 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:12.367258    5051 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 19:43:12.654431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:12.654353    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:43:12.656827 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:12.655807    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:43:12.673258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:12.673236    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:43:12.674464 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:12.674446    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:43:12.701552 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:12.701510    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7098-50265c" event={"ID":"0c3e7108-2bb3-4619-9154-9675be4b82b0","Type":"ContainerStarted","Data":"ed73477a053c34dea51ce6c7f0efa874d3e9cce822d9c0ede9802d37cf6ad034"}
Nov 03 19:43:13.712096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:13.712060    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7098-50265c" event={"ID":"0c3e7108-2bb3-4619-9154-9675be4b82b0","Type":"ContainerStarted","Data":"6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb"}
Nov 03 19:43:13.734020 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:13.733969    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7098-50265c" podStartSLOduration=1.575440987 podStartE2EDuration="2.733955449s" totalImagesPullingTime="1.158514462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:43:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:43:13.731828459 +0000 UTC m=+994.639850417" watchObservedRunningTime="2026-11-03 19:43:13.733955449 +0000 UTC m=+994.641977387"
Nov 03 19:43:14.530357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:14.530323    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7098-50265c"]
Nov 03 19:43:14.536055 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:14.536017    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7098-50265c"]
Nov 03 19:43:15.727146 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:15.727082    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7098-50265c" podUID="0c3e7108-2bb3-4619-9154-9675be4b82b0" containerName="test-curl-7098-50265c" containerID="cri-o://6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb" gracePeriod=2
Nov 03 19:43:17.957594 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:17.957563    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7098-50265c"
Nov 03 19:43:18.065819 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:18.065739    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c3e7108-2bb3-4619-9154-9675be4b82b0-kube-api-access-pv7gw\" (UniqueName: \"kubernetes.io/projected/0c3e7108-2bb3-4619-9154-9675be4b82b0-kube-api-access-pv7gw\") pod \"0c3e7108-2bb3-4619-9154-9675be4b82b0\" (UID: \"0c3e7108-2bb3-4619-9154-9675be4b82b0\") "
Nov 03 19:43:18.068092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:18.068058    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c3e7108-2bb3-4619-9154-9675be4b82b0-kube-api-access-pv7gw" pod "0c3e7108-2bb3-4619-9154-9675be4b82b0" (UID: "0c3e7108-2bb3-4619-9154-9675be4b82b0"). InnerVolumeSpecName "kube-api-access-pv7gw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:43:18.168153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:18.168119    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv7gw\" (UniqueName: \"kubernetes.io/projected/0c3e7108-2bb3-4619-9154-9675be4b82b0-kube-api-access-pv7gw\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:43:18.750918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:18.750881    5051 generic.go:356] "Generic (PLEG): container finished" podID="0c3e7108-2bb3-4619-9154-9675be4b82b0" containerID="6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb" exitCode=137
Nov 03 19:43:18.751123 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:18.750936    5051 scope.go:122] "RemoveContainer" containerID="6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb"
Nov 03 19:43:18.751123 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:18.750938    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7098-50265c"
Nov 03 19:43:18.768786 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:18.768763    5051 scope.go:122] "RemoveContainer" containerID="6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb"
Nov 03 19:43:18.769153 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:43:18.769122    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb\": container with ID starting with 6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb not found: ID does not exist" containerID="6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb"
Nov 03 19:43:18.769248 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:18.769166    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb"} err="failed to get container status \"6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb\": rpc error: code = NotFound desc = could not find container \"6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb\": container with ID starting with 6a9c40fa856811503750be828c0d0ab50d77b510cb92093cbebc426b49907dfb not found: ID does not exist"
Nov 03 19:43:18.770630 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:43:18.770601    5051 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7098-50265c\" is forbidden: User \"system:node:ip-10-0-143-212.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-212.us-east-2.compute.internal' and this object" podUID="0c3e7108-2bb3-4619-9154-9675be4b82b0" pod="opendatahub/test-curl-7098-50265c"
Nov 03 19:43:19.230436 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:43:19.230391    5051 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7098-50265c\" is forbidden: User \"system:node:ip-10-0-143-212.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-212.us-east-2.compute.internal' and this object" podUID="0c3e7108-2bb3-4619-9154-9675be4b82b0" pod="opendatahub/test-curl-7098-50265c"
Nov 03 19:43:19.235231 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:19.235201    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c3e7108-2bb3-4619-9154-9675be4b82b0" path="/var/lib/kubelet/pods/0c3e7108-2bb3-4619-9154-9675be4b82b0/volumes"
Nov 03 19:43:27.468441 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.468407    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"]
Nov 03 19:43:27.471020 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.469189    5051 cpu_manager.go:414] "containerMap: removing container" podUID="0c3e7108-2bb3-4619-9154-9675be4b82b0" containerName="test-curl-7098-50265c"
Nov 03 19:43:27.471020 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.469205    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c3e7108-2bb3-4619-9154-9675be4b82b0" containerName="test-curl-7098-50265c"
Nov 03 19:43:27.471020 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.469333    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c3e7108-2bb3-4619-9154-9675be4b82b0" containerName="test-curl-7098-50265c"
Nov 03 19:43:27.473283 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.473258    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.484459 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.484438    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-edc935cf-openshift-default-dockercfg-svk8n\""
Nov 03 19:43:27.490885 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.490859    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"]
Nov 03 19:43:27.533276 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.533233    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istiod-ca-cert\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.533472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.533364    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-podinfo\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.533472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.533426    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-data\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.533605 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.533510    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-token\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.533605 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.533552    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-envoy\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.533719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.533668    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrqds\" (UniqueName: \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-kube-api-access-vrqds\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.533719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.533701    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-socket\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.533835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.533765    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-credential-socket\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.533975 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.533957    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-certs\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.702065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.697203    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istiod-ca-cert\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.702065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.697372    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-podinfo\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.702065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.697434    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-data\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.702065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.697501    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-token\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.702065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.697551    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-envoy\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.702065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.697652    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrqds\" (UniqueName: \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-kube-api-access-vrqds\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.702065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.697681    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-socket\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.702065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.697712    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-credential-socket\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.702065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.697938    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-certs\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.702065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.698574    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-certs\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.702065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.699268    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istiod-ca-cert\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.766155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.727760    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-socket\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.766155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.728121    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-credential-socket\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.766155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.728393    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-data\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.775918 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.773678    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-token\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.793067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.784903    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-podinfo\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.793067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.786058    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-envoy\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:27.808092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:27.803759    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrqds\" (UniqueName: \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-kube-api-access-vrqds\") pod \"e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") " pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:28.094525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:28.094432    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:28.381751 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:43:28.381725    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bbed074_d2d2_4ee2_aeed_f443b924ec4c.slice/crio-961f687f17fb9f8f21661d34a31e21bae4776765ad5e1645d6e5452c719aea56 WatchSource:0}: Error finding container 961f687f17fb9f8f21661d34a31e21bae4776765ad5e1645d6e5452c719aea56: Status 404 returned error can't find the container with id 961f687f17fb9f8f21661d34a31e21bae4776765ad5e1645d6e5452c719aea56
Nov 03 19:43:28.381896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:28.381862    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"]
Nov 03 19:43:28.384798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:28.384770    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:43:28.384880 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:28.384832    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:43:28.384880 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:28.384862    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:43:28.829091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:28.829055    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" event={"ID":"7bbed074-d2d2-4ee2-aeed-f443b924ec4c","Type":"ContainerStarted","Data":"1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14"}
Nov 03 19:43:28.829506 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:28.829116    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" event={"ID":"7bbed074-d2d2-4ee2-aeed-f443b924ec4c","Type":"ContainerStarted","Data":"961f687f17fb9f8f21661d34a31e21bae4776765ad5e1645d6e5452c719aea56"}
Nov 03 19:43:28.853837 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:28.853787    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" podStartSLOduration=1.8537741140000001 podStartE2EDuration="1.853774114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:43:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:43:28.851722304 +0000 UTC m=+1009.759744261" watchObservedRunningTime="2026-11-03 19:43:28.853774114 +0000 UTC m=+1009.761796052"
Nov 03 19:43:29.094861 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:29.094756    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:29.106192 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:29.102933    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:29.838089 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:29.838062    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:29.839161 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:29.839141    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:43:32.918756 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:43:32.918717    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:43:35.456391 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:35.456342    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:43:35.531692 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:35.531649    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:35.571225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:35.571175    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:35.581168 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:35.581125    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:35.626182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:35.626133    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:35.918080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:35.853133    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:35.963063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:35.920080    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:43:36.018556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:35.931126    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:43:36.459199 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.459140    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:36.461294 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.461268    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:43:36.553066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.534126    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:36.553066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.536337    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:43:36.581631 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.581495    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:36.608011 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.607976    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:43:36.608251 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.583374    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:43:36.654772 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.654143    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:36.680067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.656090    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:43:36.680067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.656149    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:43:36.904065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.904006    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:43:36.904765 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:36.904739    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:43:37.057325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.057279    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:43:37.058875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.058841    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:43:37.063444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.063419    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:43:37.067064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.063634    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:43:37.240130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.163320    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:37.532361 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.532222    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:37.599189 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.599143    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:37.622510 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.622458    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:37.624171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.624133    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:43:37.675064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.662764    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:37.732101 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.676890    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:43:37.766327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.766291    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:37.945125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.939202    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:37.945125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.939270    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:37.945125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:37.939776    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:38.208288 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:38.203241    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:43:38.208288 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:38.203629    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:43:38.648073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:38.612656    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:38.650515 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:38.648956    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:38.721482 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:38.649002    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:38.721807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:38.650112    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:38.769441 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:38.676858    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:38.776989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:38.776947    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:38.991073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:38.987923    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:43:39.032327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:39.032278    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": context deadline exceeded"
Nov 03 19:43:39.390122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:39.386091    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:39.964074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:39.435957    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:39.999326 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:39.748195    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv" podUID="fffba9f9-cad3-4867-9e58-e7272e9b4303" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.132.0.38:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:40.000182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:39.763223    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:43:40.026119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:39.800353    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:40.026119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:39.897221    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:43:40.044109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:39.924100    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:43:40.044597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:39.948277    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:40.072119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.058516    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:40.072119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.058597    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:40.115701 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.074434    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n" podUID="c8f9579a-928c-41e0-a4ad-3438618bae19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:40.115701 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.074565    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-v4fqs" podUID="e4d4d3cd-6773-4b3b-80c6-52f5f9af92da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:40.309279 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.099981    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:40.452287 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.452224    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:40.460075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.452543    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:40.867068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.833480    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:40.867068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.833535    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:40.867068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.834264    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:40.867068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.834303    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:40.955444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:40.955390    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:43:41.038073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.024682    5051 patch_prober.go:29] interesting pod/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:41.038722 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.038665    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:41.161189 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.136389    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:43:41.228067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.137413    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:43:41.228067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.149257    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:43:41.228067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.149321    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:43:41.228067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.149342    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:43:41.228067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.199609    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:43:41.228067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.208354    5051 fsHandler.go:133] fs: disk usage and inodes count on following dirs took 1.030394007s: [/var/lib/containers/storage/overlay/2eb1521347aa28c2294cd121f98a61d898a4e60185f7965edf29085e6183f51a/diff /var/log/pods/openshift-multus_multus-additional-cni-plugins-jffnz_40a18e52-fbcd-4f04-a5f5-2ebafaea516a/kube-multus-additional-cni-plugins/0.log]; will not log again for this container unless duration exceeds 2s
Nov 03 19:43:41.352213 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.352171    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:41.490371 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:43:41.490295    5051 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.775s"
Nov 03 19:43:41.545062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.544811    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"]
Nov 03 19:43:41.845206 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:41.650977    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:42.111281 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.003888    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:42.325527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.147467    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:43:42.325527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.208607    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/health\": context deadline exceeded"
Nov 03 19:43:42.325527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.223628    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:42.325527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.238956    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:42.325527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.251839    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:42.325527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.263708    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:43:42.325527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.289075    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:43:42.362190 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.337176    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:43:42.675369 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.675277    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:42.765074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.733773    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:43:42.765074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:42.763053    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:43.172497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.168073    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:43:43.172497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.168114    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"]
Nov 03 19:43:43.183295 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.183229    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.186150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.186093    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:43.213084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.213021    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:43:43.268437 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.268403    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grf5k\" (UniqueName: \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-kube-api-access-grf5k\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.269929 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.269906    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-podinfo\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.270214 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.270198    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-data\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.270397 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.270383    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-certs\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.270560 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.270546    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istiod-ca-cert\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.270733 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.270719    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-envoy\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.270942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.270924    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-credential-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.275638 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.275591    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.275758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.275704    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-token\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.287536 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.287509    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:43:43.289054 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.289023    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:43:43.289230 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.289213    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:43:43.290588 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.290568    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:43:43.290694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.290616    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:43:43.290694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.290656    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:43:43.290694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.290688    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:43:43.377386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.377215    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-data\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.377386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.377385    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-certs\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.377649 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.377489    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istiod-ca-cert\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.377649 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.377595    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-envoy\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.377861 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.377827    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-credential-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.378033 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.378011    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.378186 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.378170    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-token\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.378271 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.378225    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grf5k\" (UniqueName: \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-kube-api-access-grf5k\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.378360 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.378275    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-podinfo\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.379703 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.378900    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-certs\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.381250 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.380969    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istiod-ca-cert\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.382625 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.382588    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-data\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.382799 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.382778    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-credential-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.383360 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.383339    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-socket\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.391881 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.391852    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-podinfo\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.392029 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.392010    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-envoy\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.399145 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.399121    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grf5k\" (UniqueName: \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-kube-api-access-grf5k\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.401294 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.401270    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-token\") pod \"e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") " pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.590456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.590426    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:43.889447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.889413    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"]
Nov 03 19:43:43.890413 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:43:43.890379    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0886ebc_5f57_4b08_b946_a7bb50bcbdd3.slice/crio-146dce3251a25f71ce39aab321fc92cba01c2482b9bba3807e34c9142bc9b39a WatchSource:0}: Error finding container 146dce3251a25f71ce39aab321fc92cba01c2482b9bba3807e34c9142bc9b39a: Status 404 returned error can't find the container with id 146dce3251a25f71ce39aab321fc92cba01c2482b9bba3807e34c9142bc9b39a
Nov 03 19:43:43.893148 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.893110    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:43:43.893257 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.893198    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:43:43.893257 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:43.893231    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:43:44.171147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:44.171030    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" event={"ID":"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3","Type":"ContainerStarted","Data":"5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18"}
Nov 03 19:43:44.171147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:44.171096    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" event={"ID":"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3","Type":"ContainerStarted","Data":"146dce3251a25f71ce39aab321fc92cba01c2482b9bba3807e34c9142bc9b39a"}
Nov 03 19:43:44.211408 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:44.211353    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" podStartSLOduration=4.211337639 podStartE2EDuration="4.211337639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:43:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:43:44.2062447 +0000 UTC m=+1025.114266639" watchObservedRunningTime="2026-11-03 19:43:44.211337639 +0000 UTC m=+1025.119359646"
Nov 03 19:43:44.590907 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:44.590872    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:44.592012 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:44.591989    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 19:43:44.592105 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:44.592050    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 19:43:45.591479 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:45.591441    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 19:43:45.591844 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:45.591513    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 19:43:46.592489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:46.592455    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 19:43:46.593127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:46.593095    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 19:43:47.591155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:47.591112    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 19:43:47.591330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:47.591191    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 19:43:48.591169 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:48.591137    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 19:43:48.591611 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:48.591217    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 19:43:49.591248 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:49.591210    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 19:43:49.591794 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:49.591276    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 19:43:50.591648 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:50.591617    5051 patch_prober.go:29] interesting pod/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 19:43:50.592032 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:50.591681    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 19:43:51.445627 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.445592    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"]
Nov 03 19:43:51.451539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.451516    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.464394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.464326    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 03 19:43:51.464394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.464335    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 03 19:43:51.464394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.464335    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 03 19:43:51.489315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.489280    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"]
Nov 03 19:43:51.518367 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.518335    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.518611 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.518589    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36d8cf52-dffc-4912-9d2c-d4b55e57a769-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.518706 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.518653    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.518769 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.518740    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9tlt\" (UniqueName: \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-kube-api-access-x9tlt\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.518831 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.518813    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.518897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.518882    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.519186 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.519165    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.565092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.565011    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8qmjp\""
Nov 03 19:43:51.620987 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.620937    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.621537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.621053    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36d8cf52-dffc-4912-9d2c-d4b55e57a769-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.621537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.621113    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.621537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.621138    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9tlt\" (UniqueName: \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-kube-api-access-x9tlt\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.621537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.621191    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.621537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.621401    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.621822 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.621666    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.622024 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.621999    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.623978 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.623950    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36d8cf52-dffc-4912-9d2c-d4b55e57a769-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.624575 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.624558    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.624956 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.624913    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.631443 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.631414    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.634923 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.634898    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9tlt\" (UniqueName: \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-kube-api-access-x9tlt\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.635620 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.635599    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxs8x\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:51.685331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.685300    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:51.685861 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.685843    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:51.687009 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.686985    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:43:51.769523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:51.769484    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:52.084232 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:43:52.084196    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d8cf52_dffc_4912_9d2c_d4b55e57a769.slice/crio-af37fd699720ab2669deef0d1bdda8f5609a2e1aa9bbbfea18374474687dee27 WatchSource:0}: Error finding container af37fd699720ab2669deef0d1bdda8f5609a2e1aa9bbbfea18374474687dee27: Status 404 returned error can't find the container with id af37fd699720ab2669deef0d1bdda8f5609a2e1aa9bbbfea18374474687dee27
Nov 03 19:43:52.100711 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:52.100679    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"]
Nov 03 19:43:52.269501 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:52.269466    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x" event={"ID":"36d8cf52-dffc-4912-9d2c-d4b55e57a769","Type":"ContainerStarted","Data":"af37fd699720ab2669deef0d1bdda8f5609a2e1aa9bbbfea18374474687dee27"}
Nov 03 19:43:55.615982 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:55.615948    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:43:55.616340 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:55.616011    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:43:56.310013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:56.309977    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x" event={"ID":"36d8cf52-dffc-4912-9d2c-d4b55e57a769","Type":"ContainerStarted","Data":"e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4"}
Nov 03 19:43:56.310281 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:56.310184    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:56.312354 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:56.312326    5051 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-xxs8x container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 19:43:56.312565 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:56.312543    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x" podUID="36d8cf52-dffc-4912-9d2c-d4b55e57a769" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 19:43:56.365542 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:56.365492    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x" podStartSLOduration=1.8385681489999999 podStartE2EDuration="5.365480025s" totalImagesPullingTime="3.526911876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:43:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:43:56.358711156 +0000 UTC m=+1037.266733100" watchObservedRunningTime="2026-11-03 19:43:56.365480025 +0000 UTC m=+1037.273501963"
Nov 03 19:43:56.822829 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:56.822794    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"]
Nov 03 19:43:56.827488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:56.827470    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:43:56.844240 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:56.844214    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"]
Nov 03 19:43:56.910119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:56.902407    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-28ef3ba9-dockercfg-vjjjq\""
Nov 03 19:43:56.910119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:56.906264    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-28ef3ba9\""
Nov 03 19:43:57.267065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:57.253855    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a49e95e-655e-4645-86ad-2ad9e707bd3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4\" (UID: \"0a49e95e-655e-4645-86ad-2ad9e707bd3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:43:57.267065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:57.253904    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qvw9\" (UniqueName: \"kubernetes.io/projected/0a49e95e-655e-4645-86ad-2ad9e707bd3f-kube-api-access-4qvw9\") pod \"payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4\" (UID: \"0a49e95e-655e-4645-86ad-2ad9e707bd3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:43:57.342996 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:57.342967    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:43:57.368409 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:57.368378    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a49e95e-655e-4645-86ad-2ad9e707bd3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4\" (UID: \"0a49e95e-655e-4645-86ad-2ad9e707bd3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:43:57.368409 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:57.368413    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qvw9\" (UniqueName: \"kubernetes.io/projected/0a49e95e-655e-4645-86ad-2ad9e707bd3f-kube-api-access-4qvw9\") pod \"payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4\" (UID: \"0a49e95e-655e-4645-86ad-2ad9e707bd3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:43:57.369547 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:57.369521    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a49e95e-655e-4645-86ad-2ad9e707bd3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4\" (UID: \"0a49e95e-655e-4645-86ad-2ad9e707bd3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:43:57.396114 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:57.396085    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qvw9\" (UniqueName: \"kubernetes.io/projected/0a49e95e-655e-4645-86ad-2ad9e707bd3f-kube-api-access-4qvw9\") pod \"payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4\" (UID: \"0a49e95e-655e-4645-86ad-2ad9e707bd3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:43:57.455749 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:57.453511    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:43:57.838950 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:43:57.838875    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a49e95e_655e_4645_86ad_2ad9e707bd3f.slice/crio-8fce66ea022e82108a231eb6508e8cba60601914d1df03ec7ad8d1612fcf24ba WatchSource:0}: Error finding container 8fce66ea022e82108a231eb6508e8cba60601914d1df03ec7ad8d1612fcf24ba: Status 404 returned error can't find the container with id 8fce66ea022e82108a231eb6508e8cba60601914d1df03ec7ad8d1612fcf24ba
Nov 03 19:43:57.844011 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:57.843973    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"]
Nov 03 19:43:58.352475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:58.352432    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4" event={"ID":"0a49e95e-655e-4645-86ad-2ad9e707bd3f","Type":"ContainerStarted","Data":"84d9c1f926b2d1636dfd28d4ac63cdcfebb743d481618a11927b91e930f45a09"}
Nov 03 19:43:58.352702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:58.352502    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4" event={"ID":"0a49e95e-655e-4645-86ad-2ad9e707bd3f","Type":"ContainerStarted","Data":"8fce66ea022e82108a231eb6508e8cba60601914d1df03ec7ad8d1612fcf24ba"}
Nov 03 19:43:58.352702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:58.352531    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:43:58.388622 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:43:58.388564    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4" podStartSLOduration=2.38854648 podStartE2EDuration="2.38854648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:43:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:43:58.385343019 +0000 UTC m=+1039.293364961" watchObservedRunningTime="2026-11-03 19:43:58.38854648 +0000 UTC m=+1039.296568420"
Nov 03 19:44:02.122410 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.122373    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"]
Nov 03 19:44:02.127389 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.127368    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:02.147264 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.147236    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"]
Nov 03 19:44:02.148316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.148293    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-9b7a388c\""
Nov 03 19:44:02.257667 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.257622    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-9b7a388c-dockercfg-qmlm8\""
Nov 03 19:44:02.284409 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.284371    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfnkk\" (UniqueName: \"kubernetes.io/projected/82ed022e-a677-447f-9581-b01040802fc2-kube-api-access-qfnkk\") pod \"payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7\" (UID: \"82ed022e-a677-447f-9581-b01040802fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:02.298072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.284947    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82ed022e-a677-447f-9581-b01040802fc2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7\" (UID: \"82ed022e-a677-447f-9581-b01040802fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:02.441522 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.425274    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82ed022e-a677-447f-9581-b01040802fc2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7\" (UID: \"82ed022e-a677-447f-9581-b01040802fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:02.441522 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.425575    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfnkk\" (UniqueName: \"kubernetes.io/projected/82ed022e-a677-447f-9581-b01040802fc2-kube-api-access-qfnkk\") pod \"payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7\" (UID: \"82ed022e-a677-447f-9581-b01040802fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:02.441522 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.427088    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82ed022e-a677-447f-9581-b01040802fc2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7\" (UID: \"82ed022e-a677-447f-9581-b01040802fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:02.515114 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.515028    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfnkk\" (UniqueName: \"kubernetes.io/projected/82ed022e-a677-447f-9581-b01040802fc2-kube-api-access-qfnkk\") pod \"payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7\" (UID: \"82ed022e-a677-447f-9581-b01040802fc2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:02.750054 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:02.750014    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:02.769829 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:44:02.769775    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:44:03.050737 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:03.050704    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"]
Nov 03 19:44:03.051462 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:44:03.051384    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ed022e_a677_447f_9581_b01040802fc2.slice/crio-2f8a8f3e34fff39641beef3bbbee6367e2781fcd809ee97a90208817c451f1e5 WatchSource:0}: Error finding container 2f8a8f3e34fff39641beef3bbbee6367e2781fcd809ee97a90208817c451f1e5: Status 404 returned error can't find the container with id 2f8a8f3e34fff39641beef3bbbee6367e2781fcd809ee97a90208817c451f1e5
Nov 03 19:44:03.489483 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:03.489442    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7" event={"ID":"82ed022e-a677-447f-9581-b01040802fc2","Type":"ContainerStarted","Data":"b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef"}
Nov 03 19:44:03.489886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:03.489504    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7" event={"ID":"82ed022e-a677-447f-9581-b01040802fc2","Type":"ContainerStarted","Data":"2f8a8f3e34fff39641beef3bbbee6367e2781fcd809ee97a90208817c451f1e5"}
Nov 03 19:44:03.489886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:03.489530    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:03.557061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:03.556316    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7" podStartSLOduration=1.5562932090000001 podStartE2EDuration="1.556293209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:44:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:44:03.51305905 +0000 UTC m=+1044.421080990" watchObservedRunningTime="2026-11-03 19:44:03.556293209 +0000 UTC m=+1044.464315150"
Nov 03 19:44:09.363294 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:09.363258    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:44:09.898312 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:09.898280    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:44:13.219065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:13.210122    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:13.219065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:13.210531    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:44:13.231083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:13.228129    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:44:13.718130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:13.715607    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:44:13.718130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:13.716601    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:44:13.844079 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:13.812155    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:13.960490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:13.956165    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:14.031293 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.031176    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:44:14.208065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.194429    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-4b5zv" podUID="fffba9f9-cad3-4867-9e58-e7272e9b4303" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.132.0.38:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:14.247067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.246162    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:14.247067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.246779    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:44:14.383480 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.248212    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:14.383480 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.248446    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:44:14.383480 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.248868    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:44:14.383480 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.248920    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:44:14.393087 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.389747    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:14.628072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.604267    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:14.721130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.719335    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:14.721130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.719804    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:44:14.721130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.720149    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:44:14.721130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.720549    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:44:14.832067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.814734    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:14.832067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:14.816468    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:44:15.041291 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:15.003137    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:44:15.041291 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:15.003685    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:44:15.138955 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:15.042105    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:15.138955 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:15.042435    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:44:15.258269 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:15.258215    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:44:15.303205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:15.269438    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:44:15.303205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:15.286193    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:15.727543 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:15.727498    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:15.728251 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:15.728221    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:44:15.891500 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:15.857533    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:44:16.012730 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:16.012630    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:44:16.063575 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:16.063529    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:16.305626 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:16.304212    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:16.307792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:16.306583    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:44:16.308548 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:16.307996    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:44:16.383098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:16.383068    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:44:16.529939 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:16.529135    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:44:16.774195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:16.771746    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:16.943071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:16.908209    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:17.033845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:17.019356    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:17.068456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:17.068410    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:17.414121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:17.378034    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:17.414121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:17.383394    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:44:17.768072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:17.754344    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:17.881191 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:17.768372    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": context deadline exceeded"
Nov 03 19:44:17.881436 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:17.825821    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:44:17.914093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:17.914024    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:18.093481 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:18.081718    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:44:18.094957 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:18.094915    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:44:18.284972 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:18.260876    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:44:18.314344 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:18.314299    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:44:18.434116 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:18.434000    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:44:18.439549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:18.439523    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:44:18.443983 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:18.443947    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:44:18.717711 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:18.716015    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:44:18.752451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:18.752418    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:44:18.798782 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:18.798752    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:44:26.960205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:26.960170    5051 generic.go:356] "Generic (PLEG): container finished" podID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerID="1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14" exitCode=0
Nov 03 19:44:26.960630 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:26.960234    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" event={"ID":"7bbed074-d2d2-4ee2-aeed-f443b924ec4c","Type":"ContainerDied","Data":"1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14"}
Nov 03 19:44:26.960713 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:26.960699    5051 scope.go:122] "RemoveContainer" containerID="1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14"
Nov 03 19:44:26.962458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:26.962430    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:44:26.962557 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:26.962499    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:44:26.962557 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:26.962543    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:44:27.977081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:27.977028    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" event={"ID":"7bbed074-d2d2-4ee2-aeed-f443b924ec4c","Type":"ContainerStarted","Data":"401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73"}
Nov 03 19:44:28.095110 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:28.095078    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:44:28.096251 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:28.096216    5051 patch_prober.go:29] interesting pod/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 03 19:44:28.096394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:28.096272    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 03 19:44:29.095204 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:29.095171    5051 patch_prober.go:29] interesting pod/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 03 19:44:29.095598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:29.095221    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 03 19:44:29.838245 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:29.838214    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:44:30.095148 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:30.095062    5051 patch_prober.go:29] interesting pod/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 03 19:44:30.095148 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:30.095112    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 03 19:44:31.094704 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:31.094672    5051 patch_prober.go:29] interesting pod/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 03 19:44:31.094902 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:31.094731    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 03 19:44:31.418717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:31.418625    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"]
Nov 03 19:44:31.419492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:31.419461    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerName="istio-proxy" containerID="cri-o://401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73" gracePeriod=30
Nov 03 19:44:32.794735 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:44:32.794692    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:44:39.090787 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.090760    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:44:39.151199 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.151169    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-token\" (UniqueName: \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-token\") pod \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") "
Nov 03 19:44:39.151370 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.151267    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-credential-socket\") pod \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") "
Nov 03 19:44:39.151370 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.151304    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-socket\") pod \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") "
Nov 03 19:44:39.151370 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.151356    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-envoy\") pod \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") "
Nov 03 19:44:39.151537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.151435    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-kube-api-access-vrqds\" (UniqueName: \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-kube-api-access-vrqds\") pod \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") "
Nov 03 19:44:39.151537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.151518    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-podinfo\") pod \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") "
Nov 03 19:44:39.151671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.151634    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-certs\") pod \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") "
Nov 03 19:44:39.151783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.151770    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-data\") pod \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") "
Nov 03 19:44:39.151898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.151640    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-credential-socket" pod "7bbed074-d2d2-4ee2-aeed-f443b924ec4c" (UID: "7bbed074-d2d2-4ee2-aeed-f443b924ec4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:44:39.152083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.152060    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-socket" pod "7bbed074-d2d2-4ee2-aeed-f443b924ec4c" (UID: "7bbed074-d2d2-4ee2-aeed-f443b924ec4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:44:39.152174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.151869    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-certs" pod "7bbed074-d2d2-4ee2-aeed-f443b924ec4c" (UID: "7bbed074-d2d2-4ee2-aeed-f443b924ec4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:44:39.152174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.151977    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istiod-ca-cert\") pod \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\" (UID: \"7bbed074-d2d2-4ee2-aeed-f443b924ec4c\") "
Nov 03 19:44:39.152323 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.152301    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istiod-ca-cert" pod "7bbed074-d2d2-4ee2-aeed-f443b924ec4c" (UID: "7bbed074-d2d2-4ee2-aeed-f443b924ec4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:44:39.152610 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.152580    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-data" pod "7bbed074-d2d2-4ee2-aeed-f443b924ec4c" (UID: "7bbed074-d2d2-4ee2-aeed-f443b924ec4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:44:39.153118 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.153089    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:39.153230 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.153121    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:39.153230 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.153137    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:39.153230 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.153152    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:39.153230 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.153166    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:39.153934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.153910    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-envoy" pod "7bbed074-d2d2-4ee2-aeed-f443b924ec4c" (UID: "7bbed074-d2d2-4ee2-aeed-f443b924ec4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:44:39.154122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.154100    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-token" pod "7bbed074-d2d2-4ee2-aeed-f443b924ec4c" (UID: "7bbed074-d2d2-4ee2-aeed-f443b924ec4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:44:39.154205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.154135    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-kube-api-access-vrqds" pod "7bbed074-d2d2-4ee2-aeed-f443b924ec4c" (UID: "7bbed074-d2d2-4ee2-aeed-f443b924ec4c"). InnerVolumeSpecName "kube-api-access-vrqds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:44:39.154205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.154148    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-podinfo" pod "7bbed074-d2d2-4ee2-aeed-f443b924ec4c" (UID: "7bbed074-d2d2-4ee2-aeed-f443b924ec4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:44:39.213907 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.213875    5051 generic.go:356] "Generic (PLEG): container finished" podID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerID="401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73" exitCode=0
Nov 03 19:44:39.214240 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.213940    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"
Nov 03 19:44:39.214512 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.213962    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" event={"ID":"7bbed074-d2d2-4ee2-aeed-f443b924ec4c","Type":"ContainerDied","Data":"401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73"}
Nov 03 19:44:39.214679 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.214654    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2" event={"ID":"7bbed074-d2d2-4ee2-aeed-f443b924ec4c","Type":"ContainerDied","Data":"961f687f17fb9f8f21661d34a31e21bae4776765ad5e1645d6e5452c719aea56"}
Nov 03 19:44:39.214792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.214727    5051 scope.go:122] "RemoveContainer" containerID="401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73"
Nov 03 19:44:39.232896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.232871    5051 scope.go:122] "RemoveContainer" containerID="1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14"
Nov 03 19:44:39.242958 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.242932    5051 scope.go:122] "RemoveContainer" containerID="401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73"
Nov 03 19:44:39.243402 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:44:39.243350    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73\": container with ID starting with 401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73 not found: ID does not exist" containerID="401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73"
Nov 03 19:44:39.243402 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.243389    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73"} err="failed to get container status \"401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73\": rpc error: code = NotFound desc = could not find container \"401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73\": container with ID starting with 401b65a197d93e65c772d49d538e3847dc7706faec1cab954da8ad95ce649a73 not found: ID does not exist"
Nov 03 19:44:39.243642 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.243412    5051 scope.go:122] "RemoveContainer" containerID="1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14"
Nov 03 19:44:39.244215 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:44:39.244188    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14\": container with ID starting with 1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14 not found: ID does not exist" containerID="1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14"
Nov 03 19:44:39.244316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.244224    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14"} err="failed to get container status \"1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14\": rpc error: code = NotFound desc = could not find container \"1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14\": container with ID starting with 1333cd58534b84b3f24891d964938c29ada47a1e1daca76b1099e06061e7bf14 not found: ID does not exist"
Nov 03 19:44:39.248393 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.248374    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"]
Nov 03 19:44:39.252402 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.252384    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-edc935cf-openshift-default-7f976977c7-q78h2"]
Nov 03 19:44:39.254630 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.254607    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:39.254710 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.254633    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:39.254710 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.254650    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrqds\" (UniqueName: \"kubernetes.io/projected/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-kube-api-access-vrqds\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:39.254710 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:39.254665    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bbed074-d2d2-4ee2-aeed-f443b924ec4c-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:41.232022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:41.231993    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" path="/var/lib/kubelet/pods/7bbed074-d2d2-4ee2-aeed-f443b924ec4c/volumes"
Nov 03 19:44:42.136150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:42.136117    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"]
Nov 03 19:44:46.245315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:46.245262    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"]
Nov 03 19:44:47.137350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:47.137313    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4" podUID="0a49e95e-655e-4645-86ad-2ad9e707bd3f" containerName="payload-pre-processing" containerID="cri-o://84d9c1f926b2d1636dfd28d4ac63cdcfebb743d481618a11927b91e930f45a09" gracePeriod=25
Nov 03 19:44:47.301277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:47.301244    5051 generic.go:356] "Generic (PLEG): container finished" podID="0a49e95e-655e-4645-86ad-2ad9e707bd3f" containerID="84d9c1f926b2d1636dfd28d4ac63cdcfebb743d481618a11927b91e930f45a09" exitCode=0
Nov 03 19:44:47.301739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:47.301326    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4" event={"ID":"0a49e95e-655e-4645-86ad-2ad9e707bd3f","Type":"ContainerDied","Data":"84d9c1f926b2d1636dfd28d4ac63cdcfebb743d481618a11927b91e930f45a09"}
Nov 03 19:44:47.476658 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:47.476627    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:44:47.531496 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:47.531462    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a49e95e-655e-4645-86ad-2ad9e707bd3f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a49e95e-655e-4645-86ad-2ad9e707bd3f-plugins-config-volume\") pod \"0a49e95e-655e-4645-86ad-2ad9e707bd3f\" (UID: \"0a49e95e-655e-4645-86ad-2ad9e707bd3f\") "
Nov 03 19:44:47.532113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:47.532087    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a49e95e-655e-4645-86ad-2ad9e707bd3f-kube-api-access-4qvw9\" (UniqueName: \"kubernetes.io/projected/0a49e95e-655e-4645-86ad-2ad9e707bd3f-kube-api-access-4qvw9\") pod \"0a49e95e-655e-4645-86ad-2ad9e707bd3f\" (UID: \"0a49e95e-655e-4645-86ad-2ad9e707bd3f\") "
Nov 03 19:44:47.534394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:47.534341    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a49e95e-655e-4645-86ad-2ad9e707bd3f-plugins-config-volume" pod "0a49e95e-655e-4645-86ad-2ad9e707bd3f" (UID: "0a49e95e-655e-4645-86ad-2ad9e707bd3f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:44:47.545578 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:47.545543    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a49e95e-655e-4645-86ad-2ad9e707bd3f-kube-api-access-4qvw9" pod "0a49e95e-655e-4645-86ad-2ad9e707bd3f" (UID: "0a49e95e-655e-4645-86ad-2ad9e707bd3f"). InnerVolumeSpecName "kube-api-access-4qvw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:44:47.635597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:47.635569    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a49e95e-655e-4645-86ad-2ad9e707bd3f-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:47.635597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:47.635595    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qvw9\" (UniqueName: \"kubernetes.io/projected/0a49e95e-655e-4645-86ad-2ad9e707bd3f-kube-api-access-4qvw9\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:48.311368 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:48.311327    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4" event={"ID":"0a49e95e-655e-4645-86ad-2ad9e707bd3f","Type":"ContainerDied","Data":"8fce66ea022e82108a231eb6508e8cba60601914d1df03ec7ad8d1612fcf24ba"}
Nov 03 19:44:48.311790 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:48.311404    5051 scope.go:122] "RemoveContainer" containerID="84d9c1f926b2d1636dfd28d4ac63cdcfebb743d481618a11927b91e930f45a09"
Nov 03 19:44:48.311790 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:48.311339    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"
Nov 03 19:44:48.341035 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:48.340982    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"]
Nov 03 19:44:48.429073 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:48.428585    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-28ef3ba9-68c46dbfcf-6bnt4"]
Nov 03 19:44:49.235766 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:49.235733    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a49e95e-655e-4645-86ad-2ad9e707bd3f" path="/var/lib/kubelet/pods/0a49e95e-655e-4645-86ad-2ad9e707bd3f/volumes"
Nov 03 19:44:51.277364 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:51.277150    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7" podUID="82ed022e-a677-447f-9581-b01040802fc2" containerName="payload-pre-processing" containerID="cri-o://b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef" gracePeriod=25
Nov 03 19:44:52.722065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.721231    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:52.770059 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.770001    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82ed022e-a677-447f-9581-b01040802fc2-kube-api-access-qfnkk\" (UniqueName: \"kubernetes.io/projected/82ed022e-a677-447f-9581-b01040802fc2-kube-api-access-qfnkk\") pod \"82ed022e-a677-447f-9581-b01040802fc2\" (UID: \"82ed022e-a677-447f-9581-b01040802fc2\") "
Nov 03 19:44:52.777062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.770251    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82ed022e-a677-447f-9581-b01040802fc2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82ed022e-a677-447f-9581-b01040802fc2-plugins-config-volume\") pod \"82ed022e-a677-447f-9581-b01040802fc2\" (UID: \"82ed022e-a677-447f-9581-b01040802fc2\") "
Nov 03 19:44:52.777062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.772129    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ed022e-a677-447f-9581-b01040802fc2-plugins-config-volume" pod "82ed022e-a677-447f-9581-b01040802fc2" (UID: "82ed022e-a677-447f-9581-b01040802fc2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:44:52.788471 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.781539    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ed022e-a677-447f-9581-b01040802fc2-kube-api-access-qfnkk" pod "82ed022e-a677-447f-9581-b01040802fc2" (UID: "82ed022e-a677-447f-9581-b01040802fc2"). InnerVolumeSpecName "kube-api-access-qfnkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:44:52.888639 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.888597    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfnkk\" (UniqueName: \"kubernetes.io/projected/82ed022e-a677-447f-9581-b01040802fc2-kube-api-access-qfnkk\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:52.888872 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.888857    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82ed022e-a677-447f-9581-b01040802fc2-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:44:52.979076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.972759    5051 generic.go:356] "Generic (PLEG): container finished" podID="82ed022e-a677-447f-9581-b01040802fc2" containerID="b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef" exitCode=0
Nov 03 19:44:52.979076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.972883    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7" event={"ID":"82ed022e-a677-447f-9581-b01040802fc2","Type":"ContainerDied","Data":"b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef"}
Nov 03 19:44:52.979076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.972934    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7" event={"ID":"82ed022e-a677-447f-9581-b01040802fc2","Type":"ContainerDied","Data":"2f8a8f3e34fff39641beef3bbbee6367e2781fcd809ee97a90208817c451f1e5"}
Nov 03 19:44:52.979076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.972961    5051 scope.go:122] "RemoveContainer" containerID="b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef"
Nov 03 19:44:52.979076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:52.973112    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"
Nov 03 19:44:53.038246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:53.031272    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"]
Nov 03 19:44:53.038246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:53.034991    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9b7a388c-77dbf5947-fm9t7"]
Nov 03 19:44:53.038246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:53.036250    5051 scope.go:122] "RemoveContainer" containerID="b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef"
Nov 03 19:44:53.038246 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:44:53.036683    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef\": container with ID starting with b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef not found: ID does not exist" containerID="b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef"
Nov 03 19:44:53.038246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:53.036719    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef"} err="failed to get container status \"b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef\": rpc error: code = NotFound desc = could not find container \"b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef\": container with ID starting with b82d9bc8b4eddb3652788f06ee1854071cebd276726458981ac1c2c5cb08d8ef not found: ID does not exist"
Nov 03 19:44:53.234964 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:44:53.234915    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82ed022e-a677-447f-9581-b01040802fc2" path="/var/lib/kubelet/pods/82ed022e-a677-447f-9581-b01040802fc2/volumes"
Nov 03 19:45:00.163779 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.163741    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"]
Nov 03 19:45:00.171814 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.171765    5051 cpu_manager.go:414] "containerMap: removing container" podUID="82ed022e-a677-447f-9581-b01040802fc2" containerName="payload-pre-processing"
Nov 03 19:45:00.172084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.172065    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82ed022e-a677-447f-9581-b01040802fc2" containerName="payload-pre-processing"
Nov 03 19:45:00.172250 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.172236    5051 cpu_manager.go:414] "containerMap: removing container" podUID="0a49e95e-655e-4645-86ad-2ad9e707bd3f" containerName="payload-pre-processing"
Nov 03 19:45:00.172360 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.172346    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a49e95e-655e-4645-86ad-2ad9e707bd3f" containerName="payload-pre-processing"
Nov 03 19:45:00.172644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.172626    5051 cpu_manager.go:414] "containerMap: removing container" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerName="istio-proxy"
Nov 03 19:45:00.172813 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.172777    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerName="istio-proxy"
Nov 03 19:45:00.173087 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.173026    5051 cpu_manager.go:414] "containerMap: removing container" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerName="istio-proxy"
Nov 03 19:45:00.173217 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.173194    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerName="istio-proxy"
Nov 03 19:45:00.173489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.173469    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerName="istio-proxy"
Nov 03 19:45:00.176091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.173499    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bbed074-d2d2-4ee2-aeed-f443b924ec4c" containerName="istio-proxy"
Nov 03 19:45:00.176091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.173518    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="82ed022e-a677-447f-9581-b01040802fc2" containerName="payload-pre-processing"
Nov 03 19:45:00.176091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.173529    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a49e95e-655e-4645-86ad-2ad9e707bd3f" containerName="payload-pre-processing"
Nov 03 19:45:00.178202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.178180    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"
Nov 03 19:45:00.187685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.187653    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"]
Nov 03 19:45:00.189389 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.189369    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dx9s4\""
Nov 03 19:45:00.212912 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.212890    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"]
Nov 03 19:45:00.218805 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.218782    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"
Nov 03 19:45:00.233296 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.233269    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"]
Nov 03 19:45:00.238528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.238503    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74b48dc2-fc14-40f9-8d74-051ef52d81ec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r\" (UID: \"74b48dc2-fc14-40f9-8d74-051ef52d81ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"
Nov 03 19:45:00.239006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.238981    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75c2h\" (UniqueName: \"kubernetes.io/projected/ed1cd587-5eda-4581-a88c-643d6103e85d-kube-api-access-75c2h\") pod \"maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd\" (UID: \"ed1cd587-5eda-4581-a88c-643d6103e85d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"
Nov 03 19:45:00.239224 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.239196    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1cd587-5eda-4581-a88c-643d6103e85d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd\" (UID: \"ed1cd587-5eda-4581-a88c-643d6103e85d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"
Nov 03 19:45:00.239719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.239542    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lg6r\" (UniqueName: \"kubernetes.io/projected/74b48dc2-fc14-40f9-8d74-051ef52d81ec-kube-api-access-7lg6r\") pod \"maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r\" (UID: \"74b48dc2-fc14-40f9-8d74-051ef52d81ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"
Nov 03 19:45:00.341961 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.341924    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74b48dc2-fc14-40f9-8d74-051ef52d81ec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r\" (UID: \"74b48dc2-fc14-40f9-8d74-051ef52d81ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"
Nov 03 19:45:00.342165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.341997    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75c2h\" (UniqueName: \"kubernetes.io/projected/ed1cd587-5eda-4581-a88c-643d6103e85d-kube-api-access-75c2h\") pod \"maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd\" (UID: \"ed1cd587-5eda-4581-a88c-643d6103e85d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"
Nov 03 19:45:00.342165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.342034    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1cd587-5eda-4581-a88c-643d6103e85d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd\" (UID: \"ed1cd587-5eda-4581-a88c-643d6103e85d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"
Nov 03 19:45:00.342165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.342098    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lg6r\" (UniqueName: \"kubernetes.io/projected/74b48dc2-fc14-40f9-8d74-051ef52d81ec-kube-api-access-7lg6r\") pod \"maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r\" (UID: \"74b48dc2-fc14-40f9-8d74-051ef52d81ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"
Nov 03 19:45:00.342880 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.342848    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1cd587-5eda-4581-a88c-643d6103e85d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd\" (UID: \"ed1cd587-5eda-4581-a88c-643d6103e85d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"
Nov 03 19:45:00.343007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.342934    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74b48dc2-fc14-40f9-8d74-051ef52d81ec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r\" (UID: \"74b48dc2-fc14-40f9-8d74-051ef52d81ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"
Nov 03 19:45:00.357394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.357369    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lg6r\" (UniqueName: \"kubernetes.io/projected/74b48dc2-fc14-40f9-8d74-051ef52d81ec-kube-api-access-7lg6r\") pod \"maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r\" (UID: \"74b48dc2-fc14-40f9-8d74-051ef52d81ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"
Nov 03 19:45:00.358029 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.358011    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75c2h\" (UniqueName: \"kubernetes.io/projected/ed1cd587-5eda-4581-a88c-643d6103e85d-kube-api-access-75c2h\") pod \"maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd\" (UID: \"ed1cd587-5eda-4581-a88c-643d6103e85d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"
Nov 03 19:45:00.500081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.499969    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"
Nov 03 19:45:00.540486 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.540443    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"
Nov 03 19:45:00.955194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.954980    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"]
Nov 03 19:45:00.967703 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:00.967674    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"]
Nov 03 19:45:01.041361 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:01.041323    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" event={"ID":"74b48dc2-fc14-40f9-8d74-051ef52d81ec","Type":"ContainerStarted","Data":"d6babca2503abe6fb88d27884eeb2059083594b59503fed0f5e54df12d984e38"}
Nov 03 19:45:01.042861 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:01.042829    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd" event={"ID":"ed1cd587-5eda-4581-a88c-643d6103e85d","Type":"ContainerStarted","Data":"4fbdc5c56d14c68118c10eec32e75792981be0bb564e781f75b33c3e16040cbb"}
Nov 03 19:45:02.779783 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:45:02.779746    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:45:05.083877 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:05.083839    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" event={"ID":"74b48dc2-fc14-40f9-8d74-051ef52d81ec","Type":"ContainerStarted","Data":"9c338a9453f3f62c30c1fd0ea162e8a1f1683ca8010716995a3ff79a1143c322"}
Nov 03 19:45:05.104620 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:05.104570    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" podStartSLOduration=1.5275726490000001 podStartE2EDuration="5.104556629s" totalImagesPullingTime="3.57698398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:45:05.101416782 +0000 UTC m=+1106.009438733" watchObservedRunningTime="2026-11-03 19:45:05.104556629 +0000 UTC m=+1106.012578567"
Nov 03 19:45:06.093606 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:06.093406    5051 generic.go:356] "Generic (PLEG): container finished" podID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerID="9c338a9453f3f62c30c1fd0ea162e8a1f1683ca8010716995a3ff79a1143c322" exitCode=60
Nov 03 19:45:06.094351 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:06.093728    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" event={"ID":"74b48dc2-fc14-40f9-8d74-051ef52d81ec","Type":"ContainerDied","Data":"9c338a9453f3f62c30c1fd0ea162e8a1f1683ca8010716995a3ff79a1143c322"}
Nov 03 19:45:06.094351 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:06.093894    5051 scope.go:122] "RemoveContainer" containerID="9c338a9453f3f62c30c1fd0ea162e8a1f1683ca8010716995a3ff79a1143c322"
Nov 03 19:45:06.096184 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:06.096151    5051 generic.go:356] "Generic (PLEG): container finished" podID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerID="6dd79f3fe3c52bfc2f5be05468dc9ee10551dc33bcde796078fba4b13f246ab2" exitCode=60
Nov 03 19:45:06.096292 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:06.096239    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd" event={"ID":"ed1cd587-5eda-4581-a88c-643d6103e85d","Type":"ContainerDied","Data":"6dd79f3fe3c52bfc2f5be05468dc9ee10551dc33bcde796078fba4b13f246ab2"}
Nov 03 19:45:06.096671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:06.096598    5051 scope.go:122] "RemoveContainer" containerID="6dd79f3fe3c52bfc2f5be05468dc9ee10551dc33bcde796078fba4b13f246ab2"
Nov 03 19:45:07.111320 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:07.111282    5051 generic.go:356] "Generic (PLEG): container finished" podID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerID="b8e4fbd56700be3ebf735f0f6e6fd0809af32170e69a5ed140ee4084847a8b00" exitCode=60
Nov 03 19:45:07.111742 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:07.111369    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd" event={"ID":"ed1cd587-5eda-4581-a88c-643d6103e85d","Type":"ContainerDied","Data":"b8e4fbd56700be3ebf735f0f6e6fd0809af32170e69a5ed140ee4084847a8b00"}
Nov 03 19:45:07.111742 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:07.111467    5051 scope.go:122] "RemoveContainer" containerID="6dd79f3fe3c52bfc2f5be05468dc9ee10551dc33bcde796078fba4b13f246ab2"
Nov 03 19:45:07.111833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:07.111759    5051 scope.go:122] "RemoveContainer" containerID="b8e4fbd56700be3ebf735f0f6e6fd0809af32170e69a5ed140ee4084847a8b00"
Nov 03 19:45:07.112216 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:45:07.112181    5051 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd_odh-ai-gateway-infra(ed1cd587-5eda-4581-a88c-643d6103e85d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d"
Nov 03 19:45:07.114649 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:07.114622    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" event={"ID":"74b48dc2-fc14-40f9-8d74-051ef52d81ec","Type":"ContainerStarted","Data":"430c934241b7b6e7a711aea23b8b8e22056b2ee7fb1b279cee7758329505954e"}
Nov 03 19:45:08.124175 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:08.124134    5051 generic.go:356] "Generic (PLEG): container finished" podID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerID="430c934241b7b6e7a711aea23b8b8e22056b2ee7fb1b279cee7758329505954e" exitCode=60
Nov 03 19:45:08.124663 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:08.124212    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" event={"ID":"74b48dc2-fc14-40f9-8d74-051ef52d81ec","Type":"ContainerDied","Data":"430c934241b7b6e7a711aea23b8b8e22056b2ee7fb1b279cee7758329505954e"}
Nov 03 19:45:08.124663 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:08.124274    5051 scope.go:122] "RemoveContainer" containerID="9c338a9453f3f62c30c1fd0ea162e8a1f1683ca8010716995a3ff79a1143c322"
Nov 03 19:45:08.124663 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:08.124612    5051 scope.go:122] "RemoveContainer" containerID="430c934241b7b6e7a711aea23b8b8e22056b2ee7fb1b279cee7758329505954e"
Nov 03 19:45:08.125084 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:45:08.125019    5051 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r_odh-ai-gateway-infra(74b48dc2-fc14-40f9-8d74-051ef52d81ec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec"
Nov 03 19:45:08.126572 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:08.126553    5051 scope.go:122] "RemoveContainer" containerID="b8e4fbd56700be3ebf735f0f6e6fd0809af32170e69a5ed140ee4084847a8b00"
Nov 03 19:45:08.126965 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:45:08.126946    5051 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd_odh-ai-gateway-infra(ed1cd587-5eda-4581-a88c-643d6103e85d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d"
Nov 03 19:45:09.137656 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:09.137627    5051 scope.go:122] "RemoveContainer" containerID="430c934241b7b6e7a711aea23b8b8e22056b2ee7fb1b279cee7758329505954e"
Nov 03 19:45:09.138032 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:45:09.137987    5051 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r_odh-ai-gateway-infra(74b48dc2-fc14-40f9-8d74-051ef52d81ec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec"
Nov 03 19:45:19.225770 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:19.225738    5051 scope.go:122] "RemoveContainer" containerID="b8e4fbd56700be3ebf735f0f6e6fd0809af32170e69a5ed140ee4084847a8b00"
Nov 03 19:45:20.221776 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:20.221748    5051 scope.go:122] "RemoveContainer" containerID="430c934241b7b6e7a711aea23b8b8e22056b2ee7fb1b279cee7758329505954e"
Nov 03 19:45:20.233804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:20.233776    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd" event={"ID":"ed1cd587-5eda-4581-a88c-643d6103e85d","Type":"ContainerStarted","Data":"75920981afcb992797001d2e4ecb70ecc80ba2a4556d83862e5f684dfc180895"}
Nov 03 19:45:20.286701 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:20.286640    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd" podStartSLOduration=16.540250301 podStartE2EDuration="20.286620645s" totalImagesPullingTime="3.746370344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:45:20.279589692 +0000 UTC m=+1121.187611675" watchObservedRunningTime="2026-11-03 19:45:20.286620645 +0000 UTC m=+1121.194642567"
Nov 03 19:45:20.922533 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:20.922428    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:45:21.247593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:21.247544    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" event={"ID":"74b48dc2-fc14-40f9-8d74-051ef52d81ec","Type":"ContainerStarted","Data":"314db0c8694436f6bbde720a143ff4208abbcc74d1d66d4aea092233b492ec76"}
Nov 03 19:45:21.250573 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:21.250537    5051 generic.go:356] "Generic (PLEG): container finished" podID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerID="75920981afcb992797001d2e4ecb70ecc80ba2a4556d83862e5f684dfc180895" exitCode=60
Nov 03 19:45:21.250708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:21.250603    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd" event={"ID":"ed1cd587-5eda-4581-a88c-643d6103e85d","Type":"ContainerDied","Data":"75920981afcb992797001d2e4ecb70ecc80ba2a4556d83862e5f684dfc180895"}
Nov 03 19:45:21.250708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:21.250674    5051 scope.go:122] "RemoveContainer" containerID="b8e4fbd56700be3ebf735f0f6e6fd0809af32170e69a5ed140ee4084847a8b00"
Nov 03 19:45:21.251114 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:21.251089    5051 scope.go:122] "RemoveContainer" containerID="75920981afcb992797001d2e4ecb70ecc80ba2a4556d83862e5f684dfc180895"
Nov 03 19:45:21.251502 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:45:21.251482    5051 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd_odh-ai-gateway-infra(ed1cd587-5eda-4581-a88c-643d6103e85d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d"
Nov 03 19:45:21.299643 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:21.299609    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"]
Nov 03 19:45:22.271194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.271149    5051 generic.go:356] "Generic (PLEG): container finished" podID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerID="314db0c8694436f6bbde720a143ff4208abbcc74d1d66d4aea092233b492ec76" exitCode=60
Nov 03 19:45:22.271666 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.271261    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" event={"ID":"74b48dc2-fc14-40f9-8d74-051ef52d81ec","Type":"ContainerDied","Data":"314db0c8694436f6bbde720a143ff4208abbcc74d1d66d4aea092233b492ec76"}
Nov 03 19:45:22.272137 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.271323    5051 scope.go:122] "RemoveContainer" containerID="430c934241b7b6e7a711aea23b8b8e22056b2ee7fb1b279cee7758329505954e"
Nov 03 19:45:22.272757 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.272730    5051 scope.go:122] "RemoveContainer" containerID="314db0c8694436f6bbde720a143ff4208abbcc74d1d66d4aea092233b492ec76"
Nov 03 19:45:22.273248 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:45:22.273213    5051 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r_odh-ai-gateway-infra(74b48dc2-fc14-40f9-8d74-051ef52d81ec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec"
Nov 03 19:45:22.302468 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.302430    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"]
Nov 03 19:45:22.559344 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.559320    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"
Nov 03 19:45:22.644064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.638821    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed1cd587-5eda-4581-a88c-643d6103e85d-kube-api-access-75c2h\" (UniqueName: \"kubernetes.io/projected/ed1cd587-5eda-4581-a88c-643d6103e85d-kube-api-access-75c2h\") pod \"ed1cd587-5eda-4581-a88c-643d6103e85d\" (UID: \"ed1cd587-5eda-4581-a88c-643d6103e85d\") "
Nov 03 19:45:22.644064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.639287    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed1cd587-5eda-4581-a88c-643d6103e85d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1cd587-5eda-4581-a88c-643d6103e85d-maas-api-service-ca\") pod \"ed1cd587-5eda-4581-a88c-643d6103e85d\" (UID: \"ed1cd587-5eda-4581-a88c-643d6103e85d\") "
Nov 03 19:45:22.644064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.641065    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed1cd587-5eda-4581-a88c-643d6103e85d-maas-api-service-ca" pod "ed1cd587-5eda-4581-a88c-643d6103e85d" (UID: "ed1cd587-5eda-4581-a88c-643d6103e85d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:45:22.704372 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.704325    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed1cd587-5eda-4581-a88c-643d6103e85d-kube-api-access-75c2h" pod "ed1cd587-5eda-4581-a88c-643d6103e85d" (UID: "ed1cd587-5eda-4581-a88c-643d6103e85d"). InnerVolumeSpecName "kube-api-access-75c2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:45:22.748864 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.748820    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1cd587-5eda-4581-a88c-643d6103e85d-maas-api-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:45:22.748864 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:22.748861    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75c2h\" (UniqueName: \"kubernetes.io/projected/ed1cd587-5eda-4581-a88c-643d6103e85d-kube-api-access-75c2h\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:45:23.289653 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.289623    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"
Nov 03 19:45:23.290094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.289678    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd" event={"ID":"ed1cd587-5eda-4581-a88c-643d6103e85d","Type":"ContainerDied","Data":"4fbdc5c56d14c68118c10eec32e75792981be0bb564e781f75b33c3e16040cbb"}
Nov 03 19:45:23.290094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.289730    5051 scope.go:122] "RemoveContainer" containerID="75920981afcb992797001d2e4ecb70ecc80ba2a4556d83862e5f684dfc180895"
Nov 03 19:45:23.330347 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.330314    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"]
Nov 03 19:45:23.334326 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.334296    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-036184-29895585-c7mfd"]
Nov 03 19:45:23.474866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.474840    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"
Nov 03 19:45:23.575125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.572223    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74b48dc2-fc14-40f9-8d74-051ef52d81ec-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74b48dc2-fc14-40f9-8d74-051ef52d81ec-maas-api-service-ca\") pod \"74b48dc2-fc14-40f9-8d74-051ef52d81ec\" (UID: \"74b48dc2-fc14-40f9-8d74-051ef52d81ec\") "
Nov 03 19:45:23.583122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.575601    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74b48dc2-fc14-40f9-8d74-051ef52d81ec-maas-api-service-ca" pod "74b48dc2-fc14-40f9-8d74-051ef52d81ec" (UID: "74b48dc2-fc14-40f9-8d74-051ef52d81ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:45:23.583122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.575846    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74b48dc2-fc14-40f9-8d74-051ef52d81ec-kube-api-access-7lg6r\" (UniqueName: \"kubernetes.io/projected/74b48dc2-fc14-40f9-8d74-051ef52d81ec-kube-api-access-7lg6r\") pod \"74b48dc2-fc14-40f9-8d74-051ef52d81ec\" (UID: \"74b48dc2-fc14-40f9-8d74-051ef52d81ec\") "
Nov 03 19:45:23.583122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.576708    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74b48dc2-fc14-40f9-8d74-051ef52d81ec-maas-api-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:45:23.583122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.579690    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74b48dc2-fc14-40f9-8d74-051ef52d81ec-kube-api-access-7lg6r" pod "74b48dc2-fc14-40f9-8d74-051ef52d81ec" (UID: "74b48dc2-fc14-40f9-8d74-051ef52d81ec"). InnerVolumeSpecName "kube-api-access-7lg6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:45:23.677956 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:23.677921    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lg6r\" (UniqueName: \"kubernetes.io/projected/74b48dc2-fc14-40f9-8d74-051ef52d81ec-kube-api-access-7lg6r\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:45:24.298462 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:24.298432    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"
Nov 03 19:45:24.298948 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:24.298438    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r" event={"ID":"74b48dc2-fc14-40f9-8d74-051ef52d81ec","Type":"ContainerDied","Data":"d6babca2503abe6fb88d27884eeb2059083594b59503fed0f5e54df12d984e38"}
Nov 03 19:45:24.298948 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:24.298589    5051 scope.go:122] "RemoveContainer" containerID="314db0c8694436f6bbde720a143ff4208abbcc74d1d66d4aea092233b492ec76"
Nov 03 19:45:24.336668 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:24.336642    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"]
Nov 03 19:45:24.342499 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:24.342473    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d8a4ea-29895585-qtr2r"]
Nov 03 19:45:25.230813 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:25.230781    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" path="/var/lib/kubelet/pods/74b48dc2-fc14-40f9-8d74-051ef52d81ec/volumes"
Nov 03 19:45:25.231283 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:25.231268    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d" path="/var/lib/kubelet/pods/ed1cd587-5eda-4581-a88c-643d6103e85d/volumes"
Nov 03 19:45:32.768710 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:45:32.768674    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:45:34.222438 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.222408    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"]
Nov 03 19:45:34.223181 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223163    5051 cpu_manager.go:414] "containerMap: removing container" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerName="cleanup"
Nov 03 19:45:34.223244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223181    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerName="cleanup"
Nov 03 19:45:34.223244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223218    5051 cpu_manager.go:414] "containerMap: removing container" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerName="cleanup"
Nov 03 19:45:34.223244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223224    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerName="cleanup"
Nov 03 19:45:34.223244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223239    5051 cpu_manager.go:414] "containerMap: removing container" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerName="cleanup"
Nov 03 19:45:34.223453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223247    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerName="cleanup"
Nov 03 19:45:34.223453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223255    5051 cpu_manager.go:414] "containerMap: removing container" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerName="cleanup"
Nov 03 19:45:34.223453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223260    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerName="cleanup"
Nov 03 19:45:34.223453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223279    5051 cpu_manager.go:414] "containerMap: removing container" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerName="cleanup"
Nov 03 19:45:34.223453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223283    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerName="cleanup"
Nov 03 19:45:34.223453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223405    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerName="cleanup"
Nov 03 19:45:34.223453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223412    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerName="cleanup"
Nov 03 19:45:34.223453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223420    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerName="cleanup"
Nov 03 19:45:34.223453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223428    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed1cd587-5eda-4581-a88c-643d6103e85d" containerName="cleanup"
Nov 03 19:45:34.223453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223435    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerName="cleanup"
Nov 03 19:45:34.223453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.223442    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerName="cleanup"
Nov 03 19:45:34.230431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.230411    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.240930 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.240902    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"]
Nov 03 19:45:34.253310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.253285    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-64454a-openshift-default-dockercfg-zhx28\""
Nov 03 19:45:34.287646 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.287619    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/701528a6-3102-4145-8c38-c31e6f80cd9f-istiod-ca-cert\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.287801 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.287781    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-socket\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.287870 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.287814    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-data\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.287981 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.287960    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5nfs\" (UniqueName: \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-kube-api-access-p5nfs\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.288032 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.287992    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-podinfo\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.288107 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.288029    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-credential-socket\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.288193 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.288177    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-envoy\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.288247 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.288234    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-token\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.288299 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.288285    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-certs\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.426511    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-socket\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.426586    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-data\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.426695    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5nfs\" (UniqueName: \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-kube-api-access-p5nfs\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.426724    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-podinfo\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.426765    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-credential-socket\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.426860    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-envoy\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.426915    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-token\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.427004    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-certs\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.427111    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/701528a6-3102-4145-8c38-c31e6f80cd9f-istiod-ca-cert\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.428117    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/701528a6-3102-4145-8c38-c31e6f80cd9f-istiod-ca-cert\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.428418    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-socket\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.428676    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-data\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.432657    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-podinfo\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.435015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.432942    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-credential-socket\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.472071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.471917    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-envoy\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.476586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.472598    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-certs\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.548159 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.538251    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5nfs\" (UniqueName: \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-kube-api-access-p5nfs\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.548159 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.538715    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-token\") pod \"e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") " pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.555062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.552159    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:34.908365 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:45:34.908334    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701528a6_3102_4145_8c38_c31e6f80cd9f.slice/crio-fdd94bc608fe1c4e9d9cf59b8d033230b62601de836e1588b8e1a1b2af720928 WatchSource:0}: Error finding container fdd94bc608fe1c4e9d9cf59b8d033230b62601de836e1588b8e1a1b2af720928: Status 404 returned error can't find the container with id fdd94bc608fe1c4e9d9cf59b8d033230b62601de836e1588b8e1a1b2af720928
Nov 03 19:45:34.908489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.908381    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"]
Nov 03 19:45:34.911267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.911233    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:45:34.911367 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.911298    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:45:34.911367 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:34.911329    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:45:35.542874 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:35.542838    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" event={"ID":"701528a6-3102-4145-8c38-c31e6f80cd9f","Type":"ContainerStarted","Data":"acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318"}
Nov 03 19:45:35.543277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:35.542894    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" event={"ID":"701528a6-3102-4145-8c38-c31e6f80cd9f","Type":"ContainerStarted","Data":"fdd94bc608fe1c4e9d9cf59b8d033230b62601de836e1588b8e1a1b2af720928"}
Nov 03 19:45:35.553275 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:35.553250    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:35.559207 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:35.559184    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:35.572282 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:35.572241    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" podStartSLOduration=1.572228449 podStartE2EDuration="1.572228449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:45:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:45:35.567585642 +0000 UTC m=+1136.475607582" watchObservedRunningTime="2026-11-03 19:45:35.572228449 +0000 UTC m=+1136.480250388"
Nov 03 19:45:36.550896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:36.550864    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:36.551994 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:36.551975    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:45:51.698585 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.698542    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"]
Nov 03 19:45:51.699780 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.699755    5051 cpu_manager.go:414] "containerMap: removing container" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerName="cleanup"
Nov 03 19:45:51.699967 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.699782    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74b48dc2-fc14-40f9-8d74-051ef52d81ec" containerName="cleanup"
Nov 03 19:45:51.715975 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.715940    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"]
Nov 03 19:45:51.716128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.716063    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.722343 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.722315    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-eae28ff0-openshift-default-dockercfg-xvxwd\""
Nov 03 19:45:51.840596 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.840557    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-token\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.840765 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.840638    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqnvd\" (UniqueName: \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-kube-api-access-xqnvd\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.840765 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.840728    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-socket\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.840846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.840819    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-envoy\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.840976 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.840958    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-data\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.841029 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.841014    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istiod-ca-cert\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.841165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.841147    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-certs\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.841336 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.841312    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-credential-socket\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.841405 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.841388    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-podinfo\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.950119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.946229    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-credential-socket\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.950119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.946329    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-podinfo\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.950119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.946470    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-token\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.950119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.946518    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqnvd\" (UniqueName: \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-kube-api-access-xqnvd\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.950119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.946563    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-socket\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.950119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.946620    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-envoy\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.950119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.946725    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-data\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.950119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.946773    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istiod-ca-cert\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.950119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.946836    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-certs\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.955162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.951723    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-credential-socket\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.970091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.965186    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istiod-ca-cert\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.970091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.965455    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-data\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.970091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.965685    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-socket\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:51.970091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.965906    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-certs\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:52.007067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:51.993777    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-podinfo\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:52.016071 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:52.007819    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-token\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:52.019085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:52.016768    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-envoy\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:52.123067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:52.119437    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqnvd\" (UniqueName: \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-kube-api-access-xqnvd\") pod \"e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") " pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:52.339701 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:52.335088    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:52.632897 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:45:52.632865    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffbc34cd_6a8c_4c45_b201_d3c5de936594.slice/crio-f826e95ccf45406e01f8df98d83679b5cb7049ffdecef8c0619a84380fc92232 WatchSource:0}: Error finding container f826e95ccf45406e01f8df98d83679b5cb7049ffdecef8c0619a84380fc92232: Status 404 returned error can't find the container with id f826e95ccf45406e01f8df98d83679b5cb7049ffdecef8c0619a84380fc92232
Nov 03 19:45:52.635008 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:52.634980    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"]
Nov 03 19:45:52.635700 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:52.635668    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:45:52.635804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:52.635744    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:45:52.635804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:52.635786    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:45:52.769194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:52.769143    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" event={"ID":"ffbc34cd-6a8c-4c45-b201-d3c5de936594","Type":"ContainerStarted","Data":"e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d"}
Nov 03 19:45:52.769674 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:52.769217    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" event={"ID":"ffbc34cd-6a8c-4c45-b201-d3c5de936594","Type":"ContainerStarted","Data":"f826e95ccf45406e01f8df98d83679b5cb7049ffdecef8c0619a84380fc92232"}
Nov 03 19:45:52.793235 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:52.793166    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" podStartSLOduration=1.7931457339999999 podStartE2EDuration="1.793145734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:45:52.792589979 +0000 UTC m=+1153.700611985" watchObservedRunningTime="2026-11-03 19:45:52.793145734 +0000 UTC m=+1153.701167674"
Nov 03 19:45:53.335773 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:53.335739    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:53.342029 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:53.342001    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:53.777221 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:53.777193    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:53.778109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:53.778088    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:45:54.309294 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:45:54.309261    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:46:02.769087 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:46:02.769010    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:46:08.246458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.246421    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"]
Nov 03 19:46:08.284387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.284346    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"]
Nov 03 19:46:08.284569 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.284464    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:08.288921 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.288891    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-eae28ff0-serving-cert\""
Nov 03 19:46:08.398798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.397279    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq7jj\" (UniqueName: \"kubernetes.io/projected/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-kube-api-access-zq7jj\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:08.398798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.397492    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-metrics-certs\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:08.398798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.397547    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-maas-api-tls\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:08.398798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.397589    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-openshift-service-ca\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:08.499423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.499332    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-metrics-certs\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:08.499423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.499391    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-maas-api-tls\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:08.499423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.499419    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-openshift-service-ca\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:08.499709 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:46:08.499541    5051 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-serving-cert: secret "maas-api-e2e-mt-eae28ff0-serving-cert" not found
Nov 03 19:46:08.499709 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:46:08.499607    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-maas-api-tls podName:585a4fb1-e6bc-4354-8c5e-e5ffc8506c61 nodeName:}" failed. No retries permitted until 2026-11-03 19:46:08.999587985 +0000 UTC m=+1169.907609932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-maas-api-tls") pod "maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn" (UID: "585a4fb1-e6bc-4354-8c5e-e5ffc8506c61") : secret "maas-api-e2e-mt-eae28ff0-serving-cert" not found
Nov 03 19:46:08.499709 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.499677    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq7jj\" (UniqueName: \"kubernetes.io/projected/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-kube-api-access-zq7jj\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:08.500362 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.500296    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-openshift-service-ca\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:08.502226 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.502203    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-metrics-certs\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:08.515086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:08.515007    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq7jj\" (UniqueName: \"kubernetes.io/projected/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-kube-api-access-zq7jj\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:09.022327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:09.022290    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-maas-api-tls\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:09.049063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:09.047521    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-maas-api-tls\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:09.210355 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:09.210320    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:09.564477 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:09.564366    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"]
Nov 03 19:46:10.073216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:10.071957    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn" event={"ID":"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61","Type":"ContainerStarted","Data":"da945557b2cc51251b213924e09c87a34f42699fce62bc40fabc6024dd1ceccf"}
Nov 03 19:46:11.084685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:11.084650    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn" event={"ID":"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61","Type":"ContainerStarted","Data":"15ee3feb6bbb0b93a9b4daf7a12062de63acdf574b58cae7033822d539ada562"}
Nov 03 19:46:11.085614 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:11.085593    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:11.109399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:11.109353    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn" podStartSLOduration=2.675295926 podStartE2EDuration="3.109340622s" totalImagesPullingTime="434.044696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:46:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:46:11.104348018 +0000 UTC m=+1172.012369985" watchObservedRunningTime="2026-11-03 19:46:11.109340622 +0000 UTC m=+1172.017362560"
Nov 03 19:46:13.372480 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.372447    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"]
Nov 03 19:46:13.378515 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.378488    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.390166 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.390138    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"]
Nov 03 19:46:13.404907 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.404882    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-istio-token\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.405177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.405154    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-data\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.405249 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.405206    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab38bfe-c49e-4031-8391-6075964ace09-istiod-ca-cert\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.405331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.405314    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.405478 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.405452    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-credential-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.405542 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.405500    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-certs\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.405664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.405589    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltl67\" (UniqueName: \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-kube-api-access-ltl67\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.405717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.405673    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab38bfe-c49e-4031-8391-6075964ace09-istio-podinfo\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.405777 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.405716    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-envoy\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.458794 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.458766    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-c54f85-openshift-default-dockercfg-w8jpj\""
Nov 03 19:46:13.507255 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.507228    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-istio-token\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.507392 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.507379    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-data\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.507441 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.507411    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab38bfe-c49e-4031-8391-6075964ace09-istiod-ca-cert\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.507483 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.507451    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.507540 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.507520    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-credential-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.507597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.507542    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-certs\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.507597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.507573    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltl67\" (UniqueName: \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-kube-api-access-ltl67\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.507699 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.507620    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab38bfe-c49e-4031-8391-6075964ace09-istio-podinfo\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.507699 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.507650    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-envoy\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.507817 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.507796    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-data\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.508162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.508130    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab38bfe-c49e-4031-8391-6075964ace09-istiod-ca-cert\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.508277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.508144    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.508277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.508226    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-credential-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.508371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.508312    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-certs\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.510139 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.510116    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab38bfe-c49e-4031-8391-6075964ace09-istio-podinfo\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.510224 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.510148    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-envoy\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.519371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.519349    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltl67\" (UniqueName: \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-kube-api-access-ltl67\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.519455 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.519378    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-istio-token\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:13.706983 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:13.706916    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:14.000289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:14.000255    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"]
Nov 03 19:46:14.000818 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:46:14.000779    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab38bfe_c49e_4031_8391_6075964ace09.slice/crio-2ba6e16dd1ed649b6264ee9fc8b407407da901ede8f24ab877892a7de2b23dc4 WatchSource:0}: Error finding container 2ba6e16dd1ed649b6264ee9fc8b407407da901ede8f24ab877892a7de2b23dc4: Status 404 returned error can't find the container with id 2ba6e16dd1ed649b6264ee9fc8b407407da901ede8f24ab877892a7de2b23dc4
Nov 03 19:46:14.003880 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:14.003846    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:46:14.003989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:14.003910    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:46:14.003989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:14.003939    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:46:14.109287 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:14.109255    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89" event={"ID":"dab38bfe-c49e-4031-8391-6075964ace09","Type":"ContainerStarted","Data":"72049b7fe571b009faf919b1c3b84d0606c1f6a75526a924aa42173a1a8d558e"}
Nov 03 19:46:14.109442 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:14.109315    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89" event={"ID":"dab38bfe-c49e-4031-8391-6075964ace09","Type":"ContainerStarted","Data":"2ba6e16dd1ed649b6264ee9fc8b407407da901ede8f24ab877892a7de2b23dc4"}
Nov 03 19:46:14.146274 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:14.146222    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89" podStartSLOduration=1.146208333 podStartE2EDuration="1.146208333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:46:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:46:14.143469916 +0000 UTC m=+1175.051491857" watchObservedRunningTime="2026-11-03 19:46:14.146208333 +0000 UTC m=+1175.054230271"
Nov 03 19:46:14.707378 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:14.707343    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:15.709386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:15.709354    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 03 19:46:15.709813 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:15.709410    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89" podUID="dab38bfe-c49e-4031-8391-6075964ace09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 03 19:46:16.708010 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:16.707976    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 03 19:46:16.708191 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:16.708027    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89" podUID="dab38bfe-c49e-4031-8391-6075964ace09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 03 19:46:17.708316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:17.708284    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 03 19:46:17.708774 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:17.708340    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89" podUID="dab38bfe-c49e-4031-8391-6075964ace09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 03 19:46:18.104566 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:18.104537    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:46:18.707585 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:18.707547    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 03 19:46:18.707795 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:18.707621    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89" podUID="dab38bfe-c49e-4031-8391-6075964ace09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 03 19:46:19.707379 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:19.707344    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 03 19:46:19.707746 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:19.707400    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89" podUID="dab38bfe-c49e-4031-8391-6075964ace09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 03 19:46:20.713697 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:20.713665    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:20.714261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:20.714143    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:20.714923 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:20.714900    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:46:32.768753 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:46:32.768717    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:46:35.601308 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:35.601276    5051 generic.go:356] "Generic (PLEG): container finished" podID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerID="acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318" exitCode=0
Nov 03 19:46:35.601705 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:35.601350    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" event={"ID":"701528a6-3102-4145-8c38-c31e6f80cd9f","Type":"ContainerDied","Data":"acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318"}
Nov 03 19:46:35.601835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:35.601819    5051 scope.go:122] "RemoveContainer" containerID="acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318"
Nov 03 19:46:35.603597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:35.603564    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:46:35.603691 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:35.603650    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:46:35.603691 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:35.603685    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:46:36.553212 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:36.553171    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:46:36.615444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:36.615397    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" event={"ID":"701528a6-3102-4145-8c38-c31e6f80cd9f","Type":"ContainerStarted","Data":"6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481"}
Nov 03 19:46:37.561030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:37.560996    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:46:37.566412 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:37.566386    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 19:46:37.566544 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:37.566435    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 19:46:38.554414 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:38.554372    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 19:46:38.555065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:38.555016    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 19:46:39.389762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:39.389736    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:46:39.395402 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:39.395374    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:46:39.556525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:39.556489    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 19:46:39.557175 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:39.557138    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 19:46:40.565248 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:40.565210    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 19:46:40.565691 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:40.565285    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 19:46:41.553570 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:41.553529    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 19:46:41.553887 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:41.553845    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 19:46:42.559108 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:42.559028    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:46:42.559501 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:42.559392    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:46:42.560166 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:42.560145    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:46:57.205602 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:57.205523    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:46:58.516818 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.516783    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"]
Nov 03 19:46:58.521300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.521282    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.566487 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.566457    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"]
Nov 03 19:46:58.798098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.797096    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-certs\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.798098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.797313    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-credential-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.798098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.797360    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.798098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.797394    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-envoy\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.798098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.797443    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-podinfo\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.798098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.797480    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64sv7\" (UniqueName: \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-kube-api-access-64sv7\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.798098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.797577    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-token\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.798098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.797720    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-data\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.798098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.797772    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istiod-ca-cert\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.904702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.904666    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-token\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.905379 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.905350    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-data\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.905571 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.905558    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istiod-ca-cert\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.905741 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.905729    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-certs\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.905975 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.905962    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-credential-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.906094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.906081    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.906188 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.906177    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-envoy\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.906297 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.906286    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-podinfo\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.906397 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.906383    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64sv7\" (UniqueName: \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-kube-api-access-64sv7\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.907168 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.907148    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-data\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.907896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.907873    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istiod-ca-cert\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.908299 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.908278    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-certs\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.908712 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.908693    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-credential-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.909147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.909113    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-socket\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.918872 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.918846    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-envoy\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.922943 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.922826    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-podinfo\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.927557 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.927534    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-token\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:58.930264 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:58.930196    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64sv7\" (UniqueName: \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-kube-api-access-64sv7\") pod \"e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") " pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:59.164472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:59.164373    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:46:59.735069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:59.729027    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"]
Nov 03 19:46:59.771292 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:46:59.771199    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda72d5d6f_7d72_4f4b_aba0_8c884a3723f8.slice/crio-339f2ee109067acdaf272165fff32259e2d843e828a6fa161457880b59143792 WatchSource:0}: Error finding container 339f2ee109067acdaf272165fff32259e2d843e828a6fa161457880b59143792: Status 404 returned error can't find the container with id 339f2ee109067acdaf272165fff32259e2d843e828a6fa161457880b59143792
Nov 03 19:46:59.793919 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:59.793111    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:46:59.793919 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:59.793209    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:46:59.793919 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:59.793253    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:46:59.919445 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:46:59.919406    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" event={"ID":"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8","Type":"ContainerStarted","Data":"339f2ee109067acdaf272165fff32259e2d843e828a6fa161457880b59143792"}
Nov 03 19:47:00.979441 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:00.979401    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" event={"ID":"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8","Type":"ContainerStarted","Data":"2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a"}
Nov 03 19:47:01.051072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:01.048870    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" podStartSLOduration=3.048849778 podStartE2EDuration="3.048849778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:47:01.022344111 +0000 UTC m=+1221.930366066" watchObservedRunningTime="2026-11-03 19:47:01.048849778 +0000 UTC m=+1221.956871722"
Nov 03 19:47:01.177366 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:01.177337    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:47:01.178329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:01.178227    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Nov 03 19:47:01.178329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:01.178283    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Nov 03 19:47:01.414124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:01.413624    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:47:01.735070 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:01.734866    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:47:01.904127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:01.904077    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:47:02.028522 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:02.025825    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:47:02.086057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:02.081124    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:47:02.127691 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:02.127644    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:47:02.168064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:02.165668    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Nov 03 19:47:02.168064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:02.165726    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Nov 03 19:47:02.297976 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:02.297876    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:47:02.361070 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:02.353836    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:47:02.425095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:02.424152    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:47:02.425095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:02.424740    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:47:02.553183 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:02.552406    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:47:02.639934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:02.639688    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:47:03.035061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:03.034423    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:47:03.085543 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:03.035532    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:47:03.362216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:03.362116    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:47:03.439527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:03.428211    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Nov 03 19:47:03.439527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:03.428266    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Nov 03 19:47:03.439527 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:47:03.430409    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:47:03.531635 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:03.531600    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:47:03.531971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:03.531954    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:47:03.532128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:03.532115    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:47:03.532741 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:03.532725    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:47:04.199217 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:04.199181    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Nov 03 19:47:04.199694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:04.199241    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Nov 03 19:47:05.172067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:05.171466    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Nov 03 19:47:05.172067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:05.171530    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Nov 03 19:47:06.168064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:06.167378    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Nov 03 19:47:06.168064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:06.167452    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Nov 03 19:47:07.165935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:07.165889    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Nov 03 19:47:07.166192 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:07.165945    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Nov 03 19:47:08.170756 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:08.170723    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:47:08.171333 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:08.171312    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:47:08.171967 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:08.171941    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:47:09.820194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:09.820158    5051 patch_prober.go:29] interesting pod/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:47:09.820662 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:09.820219    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:47:11.360377 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:11.360331    5051 generic.go:356] "Generic (PLEG): container finished" podID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerID="e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d" exitCode=0
Nov 03 19:47:11.360941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:11.360912    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" event={"ID":"ffbc34cd-6a8c-4c45-b201-d3c5de936594","Type":"ContainerDied","Data":"e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d"}
Nov 03 19:47:11.361637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:11.361582    5051 scope.go:122] "RemoveContainer" containerID="e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d"
Nov 03 19:47:11.363749 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:11.363717    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:47:11.363852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:11.363794    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:47:11.363852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:11.363830    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:47:12.379833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:12.379783    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" event={"ID":"ffbc34cd-6a8c-4c45-b201-d3c5de936594","Type":"ContainerStarted","Data":"0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7"}
Nov 03 19:47:13.335852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:13.335804    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:47:13.336518 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:13.336495    5051 patch_prober.go:29] interesting pod/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 19:47:13.336620 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:13.336540    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 19:47:14.335373 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:14.335340    5051 patch_prober.go:29] interesting pod/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 19:47:14.335766 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:14.335395    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 19:47:15.335846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:15.335803    5051 patch_prober.go:29] interesting pod/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 19:47:15.336299 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:15.335878    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 19:47:16.336307 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:16.336275    5051 patch_prober.go:29] interesting pod/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 19:47:16.336720 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:16.336336    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 19:47:17.336056 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:17.336013    5051 patch_prober.go:29] interesting pod/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 19:47:17.336225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:17.336079    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 19:47:18.427205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:18.427174    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:47:18.428725 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:18.428326    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:47:18.429394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:18.429369    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:47:20.450910 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.450368    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"]
Nov 03 19:47:20.455928 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.455900    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.470232 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.470207    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f292e03c-serving-cert\""
Nov 03 19:47:20.478704 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.478258    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"]
Nov 03 19:47:20.582161 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.582132    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a90fecd-909e-4eb2-8277-a375d93e3b98-openshift-service-ca\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.582325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.582184    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-metrics-certs\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.582325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.582292    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-maas-api-tls\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.582515 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.582502    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt9t2\" (UniqueName: \"kubernetes.io/projected/0a90fecd-909e-4eb2-8277-a375d93e3b98-kube-api-access-pt9t2\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.684573 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.684537    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt9t2\" (UniqueName: \"kubernetes.io/projected/0a90fecd-909e-4eb2-8277-a375d93e3b98-kube-api-access-pt9t2\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.684772 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.684635    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a90fecd-909e-4eb2-8277-a375d93e3b98-openshift-service-ca\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.684772 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.684697    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-metrics-certs\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.684865 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.684810    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-maas-api-tls\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.685545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.685511    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a90fecd-909e-4eb2-8277-a375d93e3b98-openshift-service-ca\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.688436 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.688399    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-metrics-certs\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.688608 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.688586    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-maas-api-tls\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.698161 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.698127    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt9t2\" (UniqueName: \"kubernetes.io/projected/0a90fecd-909e-4eb2-8277-a375d93e3b98-kube-api-access-pt9t2\") pod \"maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:20.786655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:20.786621    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:21.150825 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:21.129703    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"]
Nov 03 19:47:21.799464 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:21.797633    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c" event={"ID":"0a90fecd-909e-4eb2-8277-a375d93e3b98","Type":"ContainerStarted","Data":"db6566009b17d778533e0379da3ef8c3783a8c0bd0250e2b55fb86f99ea4521a"}
Nov 03 19:47:22.807021 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:22.806987    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c" event={"ID":"0a90fecd-909e-4eb2-8277-a375d93e3b98","Type":"ContainerStarted","Data":"768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89"}
Nov 03 19:47:22.807549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:22.807098    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:22.832379 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:22.832329    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c" podStartSLOduration=2.452422339 podStartE2EDuration="2.83231535s" totalImagesPullingTime="379.893011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:47:22.82692039 +0000 UTC m=+1243.734942341" watchObservedRunningTime="2026-11-03 19:47:22.83231535 +0000 UTC m=+1243.740337289"
Nov 03 19:47:28.821582 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:28.821546    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:47:32.769105 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:47:32.769061    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:47:40.465997 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:40.465964    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"]
Nov 03 19:47:40.466466 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:40.466198    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy" containerID="cri-o://29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099" gracePeriod=30
Nov 03 19:47:40.470899 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:40.470865    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"]
Nov 03 19:47:40.471140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:40.471110    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy" containerID="cri-o://72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b" gracePeriod=30
Nov 03 19:47:40.475843 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:40.475819    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"]
Nov 03 19:47:40.476053 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:40.476015    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy" containerID="cri-o://897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f" gracePeriod=30
Nov 03 19:47:40.481228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:40.481203    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"]
Nov 03 19:47:40.481415 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:40.481394    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerName="istio-proxy" containerID="cri-o://d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a" gracePeriod=30
Nov 03 19:47:45.621131 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.621064    5051 patch_prober.go:29] interesting pod/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 19:47:45.621811 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.621765    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 19:47:45.864168 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.864139    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:47:45.906281 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.906255    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:47:45.983472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.983438    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-kube-api-access-vq866\" (UniqueName: \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-kube-api-access-vq866\") pod \"951771a5-2df2-4a73-b5a1-34dd546f9c46\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") "
Nov 03 19:47:45.983807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.983532    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-token\" (UniqueName: \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-token\") pod \"951771a5-2df2-4a73-b5a1-34dd546f9c46\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") "
Nov 03 19:47:45.983807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.983627    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-certs\") pod \"951771a5-2df2-4a73-b5a1-34dd546f9c46\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") "
Nov 03 19:47:45.983807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.983656    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0ba4243-e8af-43f4-9296-89099991027e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0ba4243-e8af-43f4-9296-89099991027e-istiod-ca-cert\") pod \"f0ba4243-e8af-43f4-9296-89099991027e\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") "
Nov 03 19:47:45.983807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.983706    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-socket\") pod \"951771a5-2df2-4a73-b5a1-34dd546f9c46\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") "
Nov 03 19:47:45.983807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.983734    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-credential-socket\") pod \"951771a5-2df2-4a73-b5a1-34dd546f9c46\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") "
Nov 03 19:47:45.984195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.983827    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-data\") pod \"951771a5-2df2-4a73-b5a1-34dd546f9c46\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") "
Nov 03 19:47:45.984195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.983882    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-data\") pod \"f0ba4243-e8af-43f4-9296-89099991027e\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") "
Nov 03 19:47:45.984195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.983928    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-envoy\") pod \"f0ba4243-e8af-43f4-9296-89099991027e\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") "
Nov 03 19:47:45.984195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.983973    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-istio-token\" (UniqueName: \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-istio-token\") pod \"f0ba4243-e8af-43f4-9296-89099991027e\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") "
Nov 03 19:47:45.984195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.984089    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-podinfo\") pod \"951771a5-2df2-4a73-b5a1-34dd546f9c46\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") "
Nov 03 19:47:45.984195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.984187    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-socket\") pod \"f0ba4243-e8af-43f4-9296-89099991027e\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") "
Nov 03 19:47:45.984506 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.984274    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-envoy\") pod \"951771a5-2df2-4a73-b5a1-34dd546f9c46\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") "
Nov 03 19:47:45.984506 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.984347    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-credential-socket\") pod \"f0ba4243-e8af-43f4-9296-89099991027e\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") "
Nov 03 19:47:45.984506 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.984430    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/951771a5-2df2-4a73-b5a1-34dd546f9c46-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/951771a5-2df2-4a73-b5a1-34dd546f9c46-istiod-ca-cert\") pod \"951771a5-2df2-4a73-b5a1-34dd546f9c46\" (UID: \"951771a5-2df2-4a73-b5a1-34dd546f9c46\") "
Nov 03 19:47:45.984657 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.984533    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f0ba4243-e8af-43f4-9296-89099991027e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0ba4243-e8af-43f4-9296-89099991027e-istio-podinfo\") pod \"f0ba4243-e8af-43f4-9296-89099991027e\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") "
Nov 03 19:47:45.987303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.985658    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-data" pod "f0ba4243-e8af-43f4-9296-89099991027e" (UID: "f0ba4243-e8af-43f4-9296-89099991027e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:45.987303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.986002    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:45.987303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.986248    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-data" pod "951771a5-2df2-4a73-b5a1-34dd546f9c46" (UID: "951771a5-2df2-4a73-b5a1-34dd546f9c46"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:45.987303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.986440    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-certs" pod "951771a5-2df2-4a73-b5a1-34dd546f9c46" (UID: "951771a5-2df2-4a73-b5a1-34dd546f9c46"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:45.987303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.986633    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-socket" pod "f0ba4243-e8af-43f4-9296-89099991027e" (UID: "f0ba4243-e8af-43f4-9296-89099991027e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:45.987303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.986647    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-token" pod "951771a5-2df2-4a73-b5a1-34dd546f9c46" (UID: "951771a5-2df2-4a73-b5a1-34dd546f9c46"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:47:45.987303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.986891    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-credential-socket" pod "951771a5-2df2-4a73-b5a1-34dd546f9c46" (UID: "951771a5-2df2-4a73-b5a1-34dd546f9c46"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:45.987303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.986973    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0ba4243-e8af-43f4-9296-89099991027e-istiod-ca-cert" pod "f0ba4243-e8af-43f4-9296-89099991027e" (UID: "f0ba4243-e8af-43f4-9296-89099991027e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:47:45.987783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.987306    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-credential-socket" pod "f0ba4243-e8af-43f4-9296-89099991027e" (UID: "f0ba4243-e8af-43f4-9296-89099991027e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:45.987849 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.987811    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-envoy" pod "f0ba4243-e8af-43f4-9296-89099991027e" (UID: "f0ba4243-e8af-43f4-9296-89099991027e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:45.988209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.988183    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/951771a5-2df2-4a73-b5a1-34dd546f9c46-istiod-ca-cert" pod "951771a5-2df2-4a73-b5a1-34dd546f9c46" (UID: "951771a5-2df2-4a73-b5a1-34dd546f9c46"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:47:45.988310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.988290    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0ba4243-e8af-43f4-9296-89099991027e-istio-podinfo" pod "f0ba4243-e8af-43f4-9296-89099991027e" (UID: "f0ba4243-e8af-43f4-9296-89099991027e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:47:45.989213 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.989171    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-envoy" pod "951771a5-2df2-4a73-b5a1-34dd546f9c46" (UID: "951771a5-2df2-4a73-b5a1-34dd546f9c46"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:45.989317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.989280    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-socket" pod "951771a5-2df2-4a73-b5a1-34dd546f9c46" (UID: "951771a5-2df2-4a73-b5a1-34dd546f9c46"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:45.991800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.991744    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-kube-api-access-vq866" pod "951771a5-2df2-4a73-b5a1-34dd546f9c46" (UID: "951771a5-2df2-4a73-b5a1-34dd546f9c46"). InnerVolumeSpecName "kube-api-access-vq866". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:47:45.992021 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.992002    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-istio-token" pod "f0ba4243-e8af-43f4-9296-89099991027e" (UID: "f0ba4243-e8af-43f4-9296-89099991027e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:47:45.993208 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.993185    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-podinfo" pod "951771a5-2df2-4a73-b5a1-34dd546f9c46" (UID: "951771a5-2df2-4a73-b5a1-34dd546f9c46"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:47:45.995146 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:45.995125    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:47:46.000566 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.000547    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:47:46.021781 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.021695    5051 generic.go:356] "Generic (PLEG): container finished" podID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerID="72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b" exitCode=0
Nov 03 19:47:46.021781 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.021731    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" event={"ID":"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9","Type":"ContainerDied","Data":"72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b"}
Nov 03 19:47:46.021781 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.021763    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"
Nov 03 19:47:46.022378 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.021820    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6" event={"ID":"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9","Type":"ContainerDied","Data":"bf767bbcfe862a118d649d037de66b193b7120ea04f50f02d33035e5d00206c3"}
Nov 03 19:47:46.022378 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.021854    5051 scope.go:122] "RemoveContainer" containerID="72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b"
Nov 03 19:47:46.029880 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.029855    5051 generic.go:356] "Generic (PLEG): container finished" podID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerID="d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a" exitCode=0
Nov 03 19:47:46.029998 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.029909    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"
Nov 03 19:47:46.029998 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.029952    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" event={"ID":"951771a5-2df2-4a73-b5a1-34dd546f9c46","Type":"ContainerDied","Data":"d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a"}
Nov 03 19:47:46.029998 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.029990    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6" event={"ID":"951771a5-2df2-4a73-b5a1-34dd546f9c46","Type":"ContainerDied","Data":"a1f898b9455a2ed900f61dffacdb90b006b7eaf1cc4fb2351484ae7f095a7f25"}
Nov 03 19:47:46.031791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.031767    5051 generic.go:356] "Generic (PLEG): container finished" podID="f0ba4243-e8af-43f4-9296-89099991027e" containerID="29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099" exitCode=0
Nov 03 19:47:46.031893 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.031883    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"
Nov 03 19:47:46.031950 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.031905    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" event={"ID":"f0ba4243-e8af-43f4-9296-89099991027e","Type":"ContainerDied","Data":"29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099"}
Nov 03 19:47:46.032002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.031953    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb" event={"ID":"f0ba4243-e8af-43f4-9296-89099991027e","Type":"ContainerDied","Data":"2ee2dd3b7f9e5d4fa0af259aacbd13e7ec2b84dd27cc4cdebd1daa472d74f66c"}
Nov 03 19:47:46.035320 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.035294    5051 generic.go:356] "Generic (PLEG): container finished" podID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerID="897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f" exitCode=0
Nov 03 19:47:46.035398 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.035375    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" event={"ID":"96b0156a-0ade-4a21-8835-5b7ec2179330","Type":"ContainerDied","Data":"897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f"}
Nov 03 19:47:46.035456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.035410    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr" event={"ID":"96b0156a-0ade-4a21-8835-5b7ec2179330","Type":"ContainerDied","Data":"5d42986614aa730832df330f14cdf0a135a7d310c316651fcfdf01a384c04d21"}
Nov 03 19:47:46.035500 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.035479    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"
Nov 03 19:47:46.037965 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.037947    5051 scope.go:122] "RemoveContainer" containerID="4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf"
Nov 03 19:47:46.060091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.060028    5051 scope.go:122] "RemoveContainer" containerID="72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b"
Nov 03 19:47:46.063056 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:47:46.062899    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b\": container with ID starting with 72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b not found: ID does not exist" containerID="72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b"
Nov 03 19:47:46.063056 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.062940    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b"} err="failed to get container status \"72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b\": rpc error: code = NotFound desc = could not find container \"72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b\": container with ID starting with 72d761b54488099f8a44a7b4352b44d0708d2f9d9a7cde1cfe99bdee1b84640b not found: ID does not exist"
Nov 03 19:47:46.063056 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.062965    5051 scope.go:122] "RemoveContainer" containerID="4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf"
Nov 03 19:47:46.063631 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:47:46.063604    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf\": container with ID starting with 4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf not found: ID does not exist" containerID="4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf"
Nov 03 19:47:46.063720 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.063639    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf"} err="failed to get container status \"4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf\": rpc error: code = NotFound desc = could not find container \"4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf\": container with ID starting with 4321ac19b13dedeff78e14a09830fc4119693d7349e27e1480beb0f1806ecebf not found: ID does not exist"
Nov 03 19:47:46.063720 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.063680    5051 scope.go:122] "RemoveContainer" containerID="d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a"
Nov 03 19:47:46.079292 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.079264    5051 scope.go:122] "RemoveContainer" containerID="d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a"
Nov 03 19:47:46.079603 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.079583    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"]
Nov 03 19:47:46.079875 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:47:46.079754    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a\": container with ID starting with d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a not found: ID does not exist" containerID="d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a"
Nov 03 19:47:46.079875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.079786    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a"} err="failed to get container status \"d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a\": rpc error: code = NotFound desc = could not find container \"d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a\": container with ID starting with d5816c242d7d53288cb80e5fc2dd97fd8bd74aa4edbe5343f43aa1a2543cdf2a not found: ID does not exist"
Nov 03 19:47:46.079875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.079806    5051 scope.go:122] "RemoveContainer" containerID="29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099"
Nov 03 19:47:46.084931 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.084911    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-f5zj6"]
Nov 03 19:47:46.086736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.086713    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-socket\") pod \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") "
Nov 03 19:47:46.087231 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.087115    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-socket" pod "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" (UID: "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:46.087516 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.087447    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-token\" (UniqueName: \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-token\") pod \"96b0156a-0ade-4a21-8835-5b7ec2179330\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") "
Nov 03 19:47:46.087632 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.087569    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-envoy\") pod \"96b0156a-0ade-4a21-8835-5b7ec2179330\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") "
Nov 03 19:47:46.087632 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.087615    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-certs\") pod \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") "
Nov 03 19:47:46.087739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.087653    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-podinfo\") pod \"96b0156a-0ade-4a21-8835-5b7ec2179330\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") "
Nov 03 19:47:46.087791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.087756    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96b0156a-0ade-4a21-8835-5b7ec2179330-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b0156a-0ade-4a21-8835-5b7ec2179330-istiod-ca-cert\") pod \"96b0156a-0ade-4a21-8835-5b7ec2179330\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") "
Nov 03 19:47:46.087849 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.087811    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-kube-api-access-2jh88\" (UniqueName: \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-kube-api-access-2jh88\") pod \"f0ba4243-e8af-43f4-9296-89099991027e\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") "
Nov 03 19:47:46.087849 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.087841    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-data\") pod \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") "
Nov 03 19:47:46.087957 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.087937    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istiod-ca-cert\") pod \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") "
Nov 03 19:47:46.088187 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.088169    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-kube-api-access-8chrx\" (UniqueName: \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-kube-api-access-8chrx\") pod \"96b0156a-0ade-4a21-8835-5b7ec2179330\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") "
Nov 03 19:47:46.088319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.088305    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-data\") pod \"96b0156a-0ade-4a21-8835-5b7ec2179330\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") "
Nov 03 19:47:46.088437 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.088423    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-credential-socket\") pod \"96b0156a-0ade-4a21-8835-5b7ec2179330\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") "
Nov 03 19:47:46.088577 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.088558    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-kube-api-access-dd2cx\" (UniqueName: \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-kube-api-access-dd2cx\") pod \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") "
Nov 03 19:47:46.088698 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.088683    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-certs\") pod \"96b0156a-0ade-4a21-8835-5b7ec2179330\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") "
Nov 03 19:47:46.088871 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.088854    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-token\" (UniqueName: \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-token\") pod \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") "
Nov 03 19:47:46.089015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.088998    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-certs\") pod \"f0ba4243-e8af-43f4-9296-89099991027e\" (UID: \"f0ba4243-e8af-43f4-9296-89099991027e\") "
Nov 03 19:47:46.089238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.089219    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-socket\") pod \"96b0156a-0ade-4a21-8835-5b7ec2179330\" (UID: \"96b0156a-0ade-4a21-8835-5b7ec2179330\") "
Nov 03 19:47:46.089387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.089372    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-envoy\") pod \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") "
Nov 03 19:47:46.089501 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.089486    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-credential-socket\") pod \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") "
Nov 03 19:47:46.089678 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.089628    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-podinfo\") pod \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\" (UID: \"a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9\") "
Nov 03 19:47:46.090302 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.090275    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-token" pod "96b0156a-0ade-4a21-8835-5b7ec2179330" (UID: "96b0156a-0ade-4a21-8835-5b7ec2179330"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:47:46.090842 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.090817    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq866\" (UniqueName: \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-kube-api-access-vq866\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091143 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091123    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091272 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091246    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091273    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091288    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0ba4243-e8af-43f4-9296-89099991027e-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091302    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091315    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091329    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091343    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091356    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091371    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091384    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091401    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091415    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/951771a5-2df2-4a73-b5a1-34dd546f9c46-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091431    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091445    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/951771a5-2df2-4a73-b5a1-34dd546f9c46-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.091788 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.091460    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0ba4243-e8af-43f4-9296-89099991027e-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.093028 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.093006    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-data" pod "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" (UID: "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:46.093331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.093311    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-certs" pod "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" (UID: "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:46.093605 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.093586    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96b0156a-0ade-4a21-8835-5b7ec2179330-istiod-ca-cert" pod "96b0156a-0ade-4a21-8835-5b7ec2179330" (UID: "96b0156a-0ade-4a21-8835-5b7ec2179330"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:47:46.093702 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.093667    5051 scope.go:122] "RemoveContainer" containerID="29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099"
Nov 03 19:47:46.094014 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.093995    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istiod-ca-cert" pod "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" (UID: "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:47:46.094895 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.094874    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-data" pod "96b0156a-0ade-4a21-8835-5b7ec2179330" (UID: "96b0156a-0ade-4a21-8835-5b7ec2179330"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:46.095019 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:47:46.094901    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099\": container with ID starting with 29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099 not found: ID does not exist" containerID="29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099"
Nov 03 19:47:46.095169 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.095151    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-socket" pod "96b0156a-0ade-4a21-8835-5b7ec2179330" (UID: "96b0156a-0ade-4a21-8835-5b7ec2179330"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:46.095269 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.095170    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099"} err="failed to get container status \"29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099\": rpc error: code = NotFound desc = could not find container \"29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099\": container with ID starting with 29f192679f04bc021fff4f603002af4abcf5e10bf07fb7e1e48637beeee81099 not found: ID does not exist"
Nov 03 19:47:46.095446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.095430    5051 scope.go:122] "RemoveContainer" containerID="897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f"
Nov 03 19:47:46.095635 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.095260    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-kube-api-access-2jh88" pod "f0ba4243-e8af-43f4-9296-89099991027e" (UID: "f0ba4243-e8af-43f4-9296-89099991027e"). InnerVolumeSpecName "kube-api-access-2jh88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:47:46.096709 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.095493    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-credential-socket" pod "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" (UID: "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:46.096709 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.095671    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-certs" pod "f0ba4243-e8af-43f4-9296-89099991027e" (UID: "f0ba4243-e8af-43f4-9296-89099991027e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:46.096866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.095919    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-envoy" pod "96b0156a-0ade-4a21-8835-5b7ec2179330" (UID: "96b0156a-0ade-4a21-8835-5b7ec2179330"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:46.096866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.095977    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-podinfo" pod "96b0156a-0ade-4a21-8835-5b7ec2179330" (UID: "96b0156a-0ade-4a21-8835-5b7ec2179330"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:47:46.096866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.096278    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-certs" pod "96b0156a-0ade-4a21-8835-5b7ec2179330" (UID: "96b0156a-0ade-4a21-8835-5b7ec2179330"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:46.096866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.096377    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-credential-socket" pod "96b0156a-0ade-4a21-8835-5b7ec2179330" (UID: "96b0156a-0ade-4a21-8835-5b7ec2179330"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:46.097980 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.097954    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-kube-api-access-dd2cx" pod "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" (UID: "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9"). InnerVolumeSpecName "kube-api-access-dd2cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:47:46.098450 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.098433    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-envoy" pod "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" (UID: "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:47:46.100368 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.100344    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-kube-api-access-8chrx" pod "96b0156a-0ade-4a21-8835-5b7ec2179330" (UID: "96b0156a-0ade-4a21-8835-5b7ec2179330"). InnerVolumeSpecName "kube-api-access-8chrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:47:46.100368 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.100361    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-podinfo" pod "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" (UID: "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:47:46.101635 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.101612    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-token" pod "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" (UID: "a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:47:46.110424 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.110405    5051 scope.go:122] "RemoveContainer" containerID="897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f"
Nov 03 19:47:46.110699 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:47:46.110678    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f\": container with ID starting with 897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f not found: ID does not exist" containerID="897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f"
Nov 03 19:47:46.110791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.110708    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f"} err="failed to get container status \"897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f\": rpc error: code = NotFound desc = could not find container \"897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f\": container with ID starting with 897af0a1b266b4b94cda3a07c189b8b8beb2ab929dba1f5f1f571786f0391f3f not found: ID does not exist"
Nov 03 19:47:46.192753 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192719    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192753 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192747    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8chrx\" (UniqueName: \"kubernetes.io/projected/96b0156a-0ade-4a21-8835-5b7ec2179330-kube-api-access-8chrx\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192769    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192778    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192787    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd2cx\" (UniqueName: \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-kube-api-access-dd2cx\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192796    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192815    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192825    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0ba4243-e8af-43f4-9296-89099991027e-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192833    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192843    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192851    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192859    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192868    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192876    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192884    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b0156a-0ade-4a21-8835-5b7ec2179330-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192892    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b0156a-0ade-4a21-8835-5b7ec2179330-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192901    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jh88\" (UniqueName: \"kubernetes.io/projected/f0ba4243-e8af-43f4-9296-89099991027e-kube-api-access-2jh88\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.192962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.192910    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:47:46.381883 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.381844    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"]
Nov 03 19:47:46.399687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.399657    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-88km6"]
Nov 03 19:47:46.452801 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.452769    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"]
Nov 03 19:47:46.458660 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.458624    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-6hjnb"]
Nov 03 19:47:46.472072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.472026    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"]
Nov 03 19:47:46.482345 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:46.482321    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-05e217-openshift-default-59c69f4bc9-4b9dr"]
Nov 03 19:47:47.231172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:47.231143    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" path="/var/lib/kubelet/pods/951771a5-2df2-4a73-b5a1-34dd546f9c46/volumes"
Nov 03 19:47:47.231708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:47.231692    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" path="/var/lib/kubelet/pods/96b0156a-0ade-4a21-8835-5b7ec2179330/volumes"
Nov 03 19:47:47.232192 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:47.232178    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" path="/var/lib/kubelet/pods/a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9/volumes"
Nov 03 19:47:47.233023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:47.233007    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0ba4243-e8af-43f4-9296-89099991027e" path="/var/lib/kubelet/pods/f0ba4243-e8af-43f4-9296-89099991027e/volumes"
Nov 03 19:47:55.940953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.940910    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"]
Nov 03 19:47:55.942102 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942082    5051 cpu_manager.go:414] "containerMap: removing container" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy"
Nov 03 19:47:55.942177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942106    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy"
Nov 03 19:47:55.942177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942128    5051 cpu_manager.go:414] "containerMap: removing container" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy"
Nov 03 19:47:55.942177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942136    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy"
Nov 03 19:47:55.942271 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942199    5051 cpu_manager.go:414] "containerMap: removing container" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerName="istio-proxy"
Nov 03 19:47:55.942271 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942207    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerName="istio-proxy"
Nov 03 19:47:55.942271 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942236    5051 cpu_manager.go:414] "containerMap: removing container" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy"
Nov 03 19:47:55.942271 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942245    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy"
Nov 03 19:47:55.942469 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942455    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="951771a5-2df2-4a73-b5a1-34dd546f9c46" containerName="istio-proxy"
Nov 03 19:47:55.942507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942479    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy"
Nov 03 19:47:55.942507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942489    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0ba4243-e8af-43f4-9296-89099991027e" containerName="istio-proxy"
Nov 03 19:47:55.942507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942505    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy"
Nov 03 19:47:55.942601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.942514    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="96b0156a-0ade-4a21-8835-5b7ec2179330" containerName="istio-proxy"
Nov 03 19:47:55.948818 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.948800    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:55.965354 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.965328    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-29ab98-openshift-default-dockercfg-x7k8r\""
Nov 03 19:47:55.969019 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:55.968992    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"]
Nov 03 19:47:56.083525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.083487    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-envoy\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.083525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.083525    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-podinfo\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.083796 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.083554    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-certs\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.083796 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.083641    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtf2f\" (UniqueName: \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-kube-api-access-rtf2f\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.083796 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.083670    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-socket\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.083796 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.083738    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-token\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.084024 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.083847    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-credential-socket\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.084024 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.083895    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-data\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.084024 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.083938    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deae5dcc-626c-4635-a8ff-455a48be49e1-istiod-ca-cert\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.190270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.190228    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-token\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.190668 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.190644    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-credential-socket\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.190831 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.190816    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-data\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.191033 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.190978    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deae5dcc-626c-4635-a8ff-455a48be49e1-istiod-ca-cert\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.191296 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.191277    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-envoy\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.191415 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.191400    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-podinfo\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.191525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.191510    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-certs\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.191663 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.191649    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtf2f\" (UniqueName: \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-kube-api-access-rtf2f\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.191777 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.191764    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-socket\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.204356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.192463    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-socket\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.218081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.205397    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deae5dcc-626c-4635-a8ff-455a48be49e1-istiod-ca-cert\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.218081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.205913    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-credential-socket\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.218081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.206172    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-data\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.218081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.207109    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-certs\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.225961 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.225929    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-envoy\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.228965 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.228165    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-token\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.250718 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.250638    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-podinfo\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.270850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.266377    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtf2f\" (UniqueName: \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-kube-api-access-rtf2f\") pod \"e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") " pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.568164 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.568129    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:56.869822 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:47:56.869792    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeae5dcc_626c_4635_a8ff_455a48be49e1.slice/crio-271b88c550ef23dd54c58793dbb2fc04b07052216a5b92c441a28ee60e7710f7 WatchSource:0}: Error finding container 271b88c550ef23dd54c58793dbb2fc04b07052216a5b92c441a28ee60e7710f7: Status 404 returned error can't find the container with id 271b88c550ef23dd54c58793dbb2fc04b07052216a5b92c441a28ee60e7710f7
Nov 03 19:47:56.870801 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.870777    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"]
Nov 03 19:47:56.873257 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.873221    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:47:56.873332 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.873300    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:47:56.873332 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:56.873328    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:47:57.181441 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:57.181340    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" event={"ID":"deae5dcc-626c-4635-a8ff-455a48be49e1","Type":"ContainerStarted","Data":"aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e"}
Nov 03 19:47:57.181784 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:57.181452    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" event={"ID":"deae5dcc-626c-4635-a8ff-455a48be49e1","Type":"ContainerStarted","Data":"271b88c550ef23dd54c58793dbb2fc04b07052216a5b92c441a28ee60e7710f7"}
Nov 03 19:47:57.207571 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:57.207517    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" podStartSLOduration=2.20750381 podStartE2EDuration="2.20750381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:47:57.204261445 +0000 UTC m=+1278.112283413" watchObservedRunningTime="2026-11-03 19:47:57.20750381 +0000 UTC m=+1278.115525748"
Nov 03 19:47:57.568364 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:57.568334    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:57.574933 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:57.574908    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:58.189181 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:58.189151    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:47:58.190142 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:47:58.190121    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:48:02.769568 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:48:02.769532    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:48:09.236789 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.236757    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"]
Nov 03 19:48:09.237640 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.237609    5051 cpu_manager.go:414] "containerMap: removing container" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy"
Nov 03 19:48:09.237640 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.237628    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1ae873d-0a2e-4f18-ad2a-5d3544cf40f9" containerName="istio-proxy"
Nov 03 19:48:09.246986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.246963    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.252428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.252344    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"]
Nov 03 19:48:09.280989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.280959    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.281139 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.281098    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98ab7b99-b947-4052-81f6-dfe33dd18191-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.281530 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.281501    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.281685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.281594    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnvwb\" (UniqueName: \"kubernetes.io/projected/98ab7b99-b947-4052-81f6-dfe33dd18191-kube-api-access-cnvwb\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.302725 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.302705    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-29ab98-serving-cert\""
Nov 03 19:48:09.386835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.383391    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98ab7b99-b947-4052-81f6-dfe33dd18191-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.386835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.383661    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.386835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.383720    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnvwb\" (UniqueName: \"kubernetes.io/projected/98ab7b99-b947-4052-81f6-dfe33dd18191-kube-api-access-cnvwb\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.386835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.383900    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.390342 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.387999    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.390342 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.390198    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98ab7b99-b947-4052-81f6-dfe33dd18191-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.395005 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.394952    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.399532 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.399492    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnvwb\" (UniqueName: \"kubernetes.io/projected/98ab7b99-b947-4052-81f6-dfe33dd18191-kube-api-access-cnvwb\") pod \"maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.573152 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.573025    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:09.939314 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:48:09.939267    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ab7b99_b947_4052_81f6_dfe33dd18191.slice/crio-aa1f20e30de328deb1a57b7601a8620e7e8c9a9c5905cb16b02ab370402256ab WatchSource:0}: Error finding container aa1f20e30de328deb1a57b7601a8620e7e8c9a9c5905cb16b02ab370402256ab: Status 404 returned error can't find the container with id aa1f20e30de328deb1a57b7601a8620e7e8c9a9c5905cb16b02ab370402256ab
Nov 03 19:48:09.943446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:09.943403    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"]
Nov 03 19:48:10.513085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:10.513017    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b" event={"ID":"98ab7b99-b947-4052-81f6-dfe33dd18191","Type":"ContainerStarted","Data":"2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f"}
Nov 03 19:48:10.513474 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:10.513097    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b" event={"ID":"98ab7b99-b947-4052-81f6-dfe33dd18191","Type":"ContainerStarted","Data":"aa1f20e30de328deb1a57b7601a8620e7e8c9a9c5905cb16b02ab370402256ab"}
Nov 03 19:48:10.513474 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:10.513126    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:10.543303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:10.543245    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b" podStartSLOduration=1.080447601 podStartE2EDuration="1.543227664s" totalImagesPullingTime="462.780063ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:48:10.538922343 +0000 UTC m=+1291.446944297" watchObservedRunningTime="2026-11-03 19:48:10.543227664 +0000 UTC m=+1291.451249605"
Nov 03 19:48:15.845934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:15.845900    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:48:16.526732 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:16.526705    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:48:21.492963 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:21.492871    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"]
Nov 03 19:48:21.493491 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:21.493127    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn" podUID="585a4fb1-e6bc-4354-8c5e-e5ffc8506c61" containerName="maas-api" containerID="cri-o://15ee3feb6bbb0b93a9b4daf7a12062de63acdf574b58cae7033822d539ada562" gracePeriod=30
Nov 03 19:48:21.779120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:21.778824    5051 generic.go:356] "Generic (PLEG): container finished" podID="585a4fb1-e6bc-4354-8c5e-e5ffc8506c61" containerID="15ee3feb6bbb0b93a9b4daf7a12062de63acdf574b58cae7033822d539ada562" exitCode=0
Nov 03 19:48:21.779120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:21.779081    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn" event={"ID":"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61","Type":"ContainerDied","Data":"15ee3feb6bbb0b93a9b4daf7a12062de63acdf574b58cae7033822d539ada562"}
Nov 03 19:48:22.022456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.022425    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:48:22.089655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.089563    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-openshift-service-ca\") pod \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") "
Nov 03 19:48:22.089866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.089846    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-metrics-certs\" (UniqueName: \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-metrics-certs\") pod \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") "
Nov 03 19:48:22.089934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.089905    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-maas-api-tls\") pod \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") "
Nov 03 19:48:22.090122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.090103    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-kube-api-access-zq7jj\" (UniqueName: \"kubernetes.io/projected/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-kube-api-access-zq7jj\") pod \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\" (UID: \"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61\") "
Nov 03 19:48:22.091718 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.091690    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-openshift-service-ca" pod "585a4fb1-e6bc-4354-8c5e-e5ffc8506c61" (UID: "585a4fb1-e6bc-4354-8c5e-e5ffc8506c61"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:48:22.106515 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.106461    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-kube-api-access-zq7jj" pod "585a4fb1-e6bc-4354-8c5e-e5ffc8506c61" (UID: "585a4fb1-e6bc-4354-8c5e-e5ffc8506c61"). InnerVolumeSpecName "kube-api-access-zq7jj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:48:22.108863 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.108625    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-maas-api-tls" pod "585a4fb1-e6bc-4354-8c5e-e5ffc8506c61" (UID: "585a4fb1-e6bc-4354-8c5e-e5ffc8506c61"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:48:22.110981 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.110956    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-metrics-certs" pod "585a4fb1-e6bc-4354-8c5e-e5ffc8506c61" (UID: "585a4fb1-e6bc-4354-8c5e-e5ffc8506c61"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:48:22.193384 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.193347    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:48:22.193384 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.193379    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:48:22.193384 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.193389    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq7jj\" (UniqueName: \"kubernetes.io/projected/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-kube-api-access-zq7jj\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:48:22.193629 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.193399    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:48:22.787758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.787715    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"
Nov 03 19:48:22.787758 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.787738    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn" event={"ID":"585a4fb1-e6bc-4354-8c5e-e5ffc8506c61","Type":"ContainerDied","Data":"da945557b2cc51251b213924e09c87a34f42699fce62bc40fabc6024dd1ceccf"}
Nov 03 19:48:22.788305 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.787848    5051 scope.go:122] "RemoveContainer" containerID="15ee3feb6bbb0b93a9b4daf7a12062de63acdf574b58cae7033822d539ada562"
Nov 03 19:48:22.814480 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.814456    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"]
Nov 03 19:48:22.818747 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:22.818724    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-tl2nn"]
Nov 03 19:48:23.231523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:23.231490    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="585a4fb1-e6bc-4354-8c5e-e5ffc8506c61" path="/var/lib/kubelet/pods/585a4fb1-e6bc-4354-8c5e-e5ffc8506c61/volumes"
Nov 03 19:48:25.890201 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.890163    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"]
Nov 03 19:48:25.891929 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.891897    5051 cpu_manager.go:414] "containerMap: removing container" podUID="585a4fb1-e6bc-4354-8c5e-e5ffc8506c61" containerName="maas-api"
Nov 03 19:48:25.891929 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.891930    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="585a4fb1-e6bc-4354-8c5e-e5ffc8506c61" containerName="maas-api"
Nov 03 19:48:25.892261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.892240    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="585a4fb1-e6bc-4354-8c5e-e5ffc8506c61" containerName="maas-api"
Nov 03 19:48:25.898296 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.898273    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:25.919792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.916423    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"]
Nov 03 19:48:25.919792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.918573    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-25944c-openshift-default-dockercfg-9bh6l\""
Nov 03 19:48:25.955958 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.955895    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-socket\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:25.956155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.956128    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-envoy\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:25.956414 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.956387    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-certs\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:25.956488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.956465    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-credential-socket\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:25.956622 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.956596    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f06217f-17b2-4745-bdf2-6ae876890f1a-istiod-ca-cert\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:25.956721 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.956665    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-data\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:25.956721 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.956697    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6dzx\" (UniqueName: \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-kube-api-access-k6dzx\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:25.956840 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.956733    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-podinfo\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:25.956840 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:25.956798    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-token\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.063064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.058914    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-certs\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.063064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.059001    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-credential-socket\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.063064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.059123    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f06217f-17b2-4745-bdf2-6ae876890f1a-istiod-ca-cert\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.063064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.059184    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-data\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.063064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.059214    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6dzx\" (UniqueName: \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-kube-api-access-k6dzx\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.063064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.059247    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-podinfo\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.063064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.059272    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-token\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.063064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.059358    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-socket\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.063064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.059466    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-envoy\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.075150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.070212    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-data\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.075150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.070518    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-certs\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.075150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.070755    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-credential-socket\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.115068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.110250    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-socket\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.123064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.118532    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f06217f-17b2-4745-bdf2-6ae876890f1a-istiod-ca-cert\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.183114 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.178733    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-token\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.200064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.193972    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6dzx\" (UniqueName: \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-kube-api-access-k6dzx\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.200064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.197235    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-envoy\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.211060 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.201814    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-podinfo\") pod \"e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") " pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.235062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.234311    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:26.544278 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.544247    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"]
Nov 03 19:48:26.546202 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:48:26.546175    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f06217f_17b2_4745_bdf2_6ae876890f1a.slice/crio-81f09ea5d1343fd6fe494ece60c13a0526af5bedce636135f801d52d6deaa711 WatchSource:0}: Error finding container 81f09ea5d1343fd6fe494ece60c13a0526af5bedce636135f801d52d6deaa711: Status 404 returned error can't find the container with id 81f09ea5d1343fd6fe494ece60c13a0526af5bedce636135f801d52d6deaa711
Nov 03 19:48:26.549654 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.549617    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:48:26.549751 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.549691    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:48:26.549751 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.549730    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:48:26.830632 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.830528    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" event={"ID":"1f06217f-17b2-4745-bdf2-6ae876890f1a","Type":"ContainerStarted","Data":"4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2"}
Nov 03 19:48:26.830795 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.830639    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" event={"ID":"1f06217f-17b2-4745-bdf2-6ae876890f1a","Type":"ContainerStarted","Data":"81f09ea5d1343fd6fe494ece60c13a0526af5bedce636135f801d52d6deaa711"}
Nov 03 19:48:26.859681 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:26.859618    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" podStartSLOduration=1.859599467 podStartE2EDuration="1.859599467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:48:26.854406904 +0000 UTC m=+1307.762428877" watchObservedRunningTime="2026-11-03 19:48:26.859599467 +0000 UTC m=+1307.767621407"
Nov 03 19:48:27.235659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:27.235629    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:27.242163 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:27.242136    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:27.838212 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:27.838181    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:27.839149 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:27.839129    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:48:32.769313 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:48:32.769270    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:48:34.647600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.647564    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"]
Nov 03 19:48:34.654465 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.654439    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.661402 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.661375    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-eae28ff0-serving-cert\""
Nov 03 19:48:34.671351 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.671314    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"]
Nov 03 19:48:34.738442 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.738411    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44c09d6-0832-49a5-bd7d-f7cae6376d43-openshift-service-ca\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.738672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.738449    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-metrics-certs\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.738672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.738533    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-maas-api-tls\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.738672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.738555    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztt5m\" (UniqueName: \"kubernetes.io/projected/e44c09d6-0832-49a5-bd7d-f7cae6376d43-kube-api-access-ztt5m\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.843057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.841470    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44c09d6-0832-49a5-bd7d-f7cae6376d43-openshift-service-ca\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.843057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.841579    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-metrics-certs\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.843057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.841830    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-maas-api-tls\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.843057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.841872    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztt5m\" (UniqueName: \"kubernetes.io/projected/e44c09d6-0832-49a5-bd7d-f7cae6376d43-kube-api-access-ztt5m\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.851063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.843544    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44c09d6-0832-49a5-bd7d-f7cae6376d43-openshift-service-ca\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.859097 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.856154    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-metrics-certs\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.872140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.870450    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztt5m\" (UniqueName: \"kubernetes.io/projected/e44c09d6-0832-49a5-bd7d-f7cae6376d43-kube-api-access-ztt5m\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.872140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.871489    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-maas-api-tls\") pod \"maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:34.983987 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:34.983946    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:35.322067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:35.314995    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"]
Nov 03 19:48:35.344167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:35.335320    5051 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 19:48:35.926371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:35.926329    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm" event={"ID":"e44c09d6-0832-49a5-bd7d-f7cae6376d43","Type":"ContainerStarted","Data":"af8453d030844bae32593b7e59f4fe9b365b0bd1d1dd4b2779c082501708a2e9"}
Nov 03 19:48:36.935930 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:36.935893    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm" event={"ID":"e44c09d6-0832-49a5-bd7d-f7cae6376d43","Type":"ContainerStarted","Data":"f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c"}
Nov 03 19:48:36.936352 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:36.935971    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:36.987128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:36.987073    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm" podStartSLOduration=2.437539127 podStartE2EDuration="2.987054472s" totalImagesPullingTime="549.515345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:48:36.98347199 +0000 UTC m=+1317.891493930" watchObservedRunningTime="2026-11-03 19:48:36.987054472 +0000 UTC m=+1317.895076409"
Nov 03 19:48:41.041756 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.041721    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"]
Nov 03 19:48:41.047158 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.047134    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.056091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.056016    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-25944c-serving-cert\""
Nov 03 19:48:41.062896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.062865    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"]
Nov 03 19:48:41.183399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.183366    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.183572 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.183528    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x94ft\" (UniqueName: \"kubernetes.io/projected/8311401f-491c-43a6-a854-d3ae1f3255ae-kube-api-access-x94ft\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.183625 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.183581    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8311401f-491c-43a6-a854-d3ae1f3255ae-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.183674 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.183662    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.285737 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.285698    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.285997 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.285970    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.286203 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.286176    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x94ft\" (UniqueName: \"kubernetes.io/projected/8311401f-491c-43a6-a854-d3ae1f3255ae-kube-api-access-x94ft\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.286278 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.286244    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8311401f-491c-43a6-a854-d3ae1f3255ae-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.286957 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.286928    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8311401f-491c-43a6-a854-d3ae1f3255ae-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.291075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.289141    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.291075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.289878    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.312406 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.312378    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x94ft\" (UniqueName: \"kubernetes.io/projected/8311401f-491c-43a6-a854-d3ae1f3255ae-kube-api-access-x94ft\") pod \"maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.395945 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.395916    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:41.714829 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.693786    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"]
Nov 03 19:48:41.987283 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:41.987194    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g" event={"ID":"8311401f-491c-43a6-a854-d3ae1f3255ae","Type":"ContainerStarted","Data":"6c753c8e104f41983e0cd0753bff14053345feaaa91a3584f6ea4f282116abfe"}
Nov 03 19:48:42.952621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:42.952583    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:48:43.003906 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:43.003862    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g" event={"ID":"8311401f-491c-43a6-a854-d3ae1f3255ae","Type":"ContainerStarted","Data":"bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264"}
Nov 03 19:48:43.005227 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:43.005199    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:43.052652 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:43.052589    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g" podStartSLOduration=1.513419758 podStartE2EDuration="2.052569736s" totalImagesPullingTime="539.149978ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:48:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:48:43.048661613 +0000 UTC m=+1323.956683578" watchObservedRunningTime="2026-11-03 19:48:43.052569736 +0000 UTC m=+1323.960591678"
Nov 03 19:48:50.026502 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:50.026467    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:48:56.524923 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.524884    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"]
Nov 03 19:48:56.532100 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.532073    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.546661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.546401    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"]
Nov 03 19:48:56.553320 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.553297    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-10e551-openshift-default-dockercfg-7hkw6\""
Nov 03 19:48:56.696096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.679338    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-certs\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.696096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.679390    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-istio-token\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.696096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.679499    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-data\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.696096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.679564    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-socket\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.696096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.679784    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls9tg\" (UniqueName: \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-kube-api-access-ls9tg\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.696096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.679820    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-envoy\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.696096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.679888    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262d3eee-5594-4d75-83e2-1eb72166f710-istiod-ca-cert\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.696096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.679928    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262d3eee-5594-4d75-83e2-1eb72166f710-istio-podinfo\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.696096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.679997    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-credential-socket\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.870153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.868605    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls9tg\" (UniqueName: \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-kube-api-access-ls9tg\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.870153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.868664    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-envoy\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.870153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.868731    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262d3eee-5594-4d75-83e2-1eb72166f710-istiod-ca-cert\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.870153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.868769    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262d3eee-5594-4d75-83e2-1eb72166f710-istio-podinfo\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.870153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.868833    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-credential-socket\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.870153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.868931    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-certs\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.870153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.868967    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-istio-token\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.870153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.869081    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-data\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.870153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.869150    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-socket\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.870153 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.869743    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-socket\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.879682 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.879650    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-envoy\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.885128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.880866    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-certs\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.885128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.883533    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-credential-socket\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.885434 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.885380    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262d3eee-5594-4d75-83e2-1eb72166f710-istiod-ca-cert\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.890086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.888760    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls9tg\" (UniqueName: \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-kube-api-access-ls9tg\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.906093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.892836    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-data\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.906093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.902843    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-istio-token\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:56.906093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:56.904586    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262d3eee-5594-4d75-83e2-1eb72166f710-istio-podinfo\") pod \"e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") " pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:57.159914 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:57.159834    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:57.556280 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:48:57.556249    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod262d3eee_5594_4d75_83e2_1eb72166f710.slice/crio-7848d3d5e75b1e8771e69092601b982fae3204a9371eb517244f5bd075d4822a WatchSource:0}: Error finding container 7848d3d5e75b1e8771e69092601b982fae3204a9371eb517244f5bd075d4822a: Status 404 returned error can't find the container with id 7848d3d5e75b1e8771e69092601b982fae3204a9371eb517244f5bd075d4822a
Nov 03 19:48:57.556693 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:57.556588    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"]
Nov 03 19:48:57.559310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:57.559279    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:48:57.559386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:57.559344    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:48:57.559386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:57.559372    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:48:58.138086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:58.138023    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" event={"ID":"262d3eee-5594-4d75-83e2-1eb72166f710","Type":"ContainerStarted","Data":"eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6"}
Nov 03 19:48:58.138269 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:58.138116    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" event={"ID":"262d3eee-5594-4d75-83e2-1eb72166f710","Type":"ContainerStarted","Data":"7848d3d5e75b1e8771e69092601b982fae3204a9371eb517244f5bd075d4822a"}
Nov 03 19:48:58.160606 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:58.160580    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:58.169792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:58.169740    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" podStartSLOduration=2.169722869 podStartE2EDuration="2.169722869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:48:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:48:58.164176947 +0000 UTC m=+1339.072198904" watchObservedRunningTime="2026-11-03 19:48:58.169722869 +0000 UTC m=+1339.077744810"
Nov 03 19:48:58.170785 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:58.170757    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:59.146342 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:59.146308    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:48:59.147329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:48:59.147306    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:49:02.770233 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:02.770197    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:49:10.701245 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:10.701199    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"]
Nov 03 19:49:10.701770 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:10.701426    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy" containerID="cri-o://676ec120c450ea6af22731af7fe1bc5486c1e7094851152feee20da2e30a282c" gracePeriod=30
Nov 03 19:49:10.706678 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:10.706650    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"]
Nov 03 19:49:10.706893 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:10.706867    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy" containerID="cri-o://5dfa8d9201e34b7f70a7a97f0e4ef240423af00735575d11aee5a3d143785bb2" gracePeriod=30
Nov 03 19:49:10.713500 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:10.713472    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"]
Nov 03 19:49:10.713675 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:10.713655    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy" containerID="cri-o://661af47b265a1d0cb28fbdaf3584e7d1a4c30ef1997e65615060913ef6da2c27" gracePeriod=30
Nov 03 19:49:11.084749 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.084712    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"]
Nov 03 19:49:11.084974 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.084915    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy" containerID="cri-o://e1111be2b9ba5609b16c755aecc6ebf6204b78f5552708b95075be7e7d10516b" gracePeriod=30
Nov 03 19:49:11.089804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.089773    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"]
Nov 03 19:49:11.090003 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.089977    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy" containerID="cri-o://16f478f6fb9ed866513d786dfc713cba63b996b50a99f414dd7e70c9db0a5122" gracePeriod=30
Nov 03 19:49:11.094617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.094591    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"]
Nov 03 19:49:11.094802 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.094770    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy" containerID="cri-o://d265ca7b3744977dd923ff9b62d094379a291cfea0cd4318203a814d655116fe" gracePeriod=30
Nov 03 19:49:11.098524 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.098501    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"]
Nov 03 19:49:11.099882 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.098758    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy" containerID="cri-o://36a498222fbedac613e1f161279f33dd21edd6df044cfdca5b3a38ed70d4ea62" gracePeriod=30
Nov 03 19:49:11.253572 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.253540    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"]
Nov 03 19:49:11.253800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.253775    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerName="istio-proxy" containerID="cri-o://09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4" gracePeriod=30
Nov 03 19:49:11.263091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.259132    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"]
Nov 03 19:49:11.263091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.259344    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy" containerID="cri-o://174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908" gracePeriod=30
Nov 03 19:49:11.263413 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.263384    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"]
Nov 03 19:49:11.263599 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.263575    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerName="istio-proxy" containerID="cri-o://3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc" gracePeriod=30
Nov 03 19:49:11.268642 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.268617    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"]
Nov 03 19:49:11.268803 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:11.268782    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy" containerID="cri-o://8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f" gracePeriod=30
Nov 03 19:49:12.026591 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.026542    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"]
Nov 03 19:49:12.036141 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.036112    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.051119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.047610    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-10e551-serving-cert\""
Nov 03 19:49:12.051119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.049508    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"]
Nov 03 19:49:12.203061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.201408    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mslkc\" (UniqueName: \"kubernetes.io/projected/92cd8215-515e-43e5-94eb-99ef8add9480-kube-api-access-mslkc\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.203061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.201596    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92cd8215-515e-43e5-94eb-99ef8add9480-openshift-service-ca\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.203061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.201739    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-maas-api-tls\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.203061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.201808    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-metrics-certs\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.304916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.304823    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-maas-api-tls\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.304916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.304904    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-metrics-certs\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.305157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.305088    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mslkc\" (UniqueName: \"kubernetes.io/projected/92cd8215-515e-43e5-94eb-99ef8add9480-kube-api-access-mslkc\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.305280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.305263    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92cd8215-515e-43e5-94eb-99ef8add9480-openshift-service-ca\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.306015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.305986    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92cd8215-515e-43e5-94eb-99ef8add9480-openshift-service-ca\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.307663 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.307641    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-metrics-certs\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.307771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.307693    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-maas-api-tls\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.321392 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.321362    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mslkc\" (UniqueName: \"kubernetes.io/projected/92cd8215-515e-43e5-94eb-99ef8add9480-kube-api-access-mslkc\") pod \"maas-api-e2e-api-a-10e551-554c79698d-2lq2t\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.357945 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.357918    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:12.698522 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:12.698489    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"]
Nov 03 19:49:13.377460 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:13.377427    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t" event={"ID":"92cd8215-515e-43e5-94eb-99ef8add9480","Type":"ContainerStarted","Data":"edee234d46d0b9fb3b4a2d17962513299417ea82874ff8db920ecfba343b41f0"}
Nov 03 19:49:14.386339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:14.386304    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t" event={"ID":"92cd8215-515e-43e5-94eb-99ef8add9480","Type":"ContainerStarted","Data":"067cc6293c6bd2f897baf1f35faca96c7a38af091f4364281471812d05399ae5"}
Nov 03 19:49:14.386746 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:14.386397    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:14.408900 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:14.408858    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t" podStartSLOduration=1.892121257 podStartE2EDuration="2.408845947s" totalImagesPullingTime="516.72469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:14.405838387 +0000 UTC m=+1355.313860347" watchObservedRunningTime="2026-11-03 19:49:14.408845947 +0000 UTC m=+1355.316867886"
Nov 03 19:49:15.437384 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:15.437347    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"]
Nov 03 19:49:15.438295 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:15.438260    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm" podUID="e44c09d6-0832-49a5-bd7d-f7cae6376d43" containerName="maas-api" containerID="cri-o://f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c" gracePeriod=30
Nov 03 19:49:16.092302 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.091963    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:49:16.131144 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.130810    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-metrics-certs\") pod \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") "
Nov 03 19:49:16.131144 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.130964    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e44c09d6-0832-49a5-bd7d-f7cae6376d43-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44c09d6-0832-49a5-bd7d-f7cae6376d43-openshift-service-ca\") pod \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") "
Nov 03 19:49:16.131144 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.131012    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-maas-api-tls\") pod \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") "
Nov 03 19:49:16.132854 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.131530    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e44c09d6-0832-49a5-bd7d-f7cae6376d43-kube-api-access-ztt5m\" (UniqueName: \"kubernetes.io/projected/e44c09d6-0832-49a5-bd7d-f7cae6376d43-kube-api-access-ztt5m\") pod \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\" (UID: \"e44c09d6-0832-49a5-bd7d-f7cae6376d43\") "
Nov 03 19:49:16.134737 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.134473    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e44c09d6-0832-49a5-bd7d-f7cae6376d43-openshift-service-ca" pod "e44c09d6-0832-49a5-bd7d-f7cae6376d43" (UID: "e44c09d6-0832-49a5-bd7d-f7cae6376d43"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:16.137248 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.137216    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e44c09d6-0832-49a5-bd7d-f7cae6376d43-kube-api-access-ztt5m" pod "e44c09d6-0832-49a5-bd7d-f7cae6376d43" (UID: "e44c09d6-0832-49a5-bd7d-f7cae6376d43"). InnerVolumeSpecName "kube-api-access-ztt5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:16.138673 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.138641    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-maas-api-tls" pod "e44c09d6-0832-49a5-bd7d-f7cae6376d43" (UID: "e44c09d6-0832-49a5-bd7d-f7cae6376d43"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:49:16.163194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.161355    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-metrics-certs" pod "e44c09d6-0832-49a5-bd7d-f7cae6376d43" (UID: "e44c09d6-0832-49a5-bd7d-f7cae6376d43"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:49:16.235724 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.234155    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44c09d6-0832-49a5-bd7d-f7cae6376d43-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.235724 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.234178    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.235724 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.234189    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztt5m\" (UniqueName: \"kubernetes.io/projected/e44c09d6-0832-49a5-bd7d-f7cae6376d43-kube-api-access-ztt5m\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.235724 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.234204    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e44c09d6-0832-49a5-bd7d-f7cae6376d43-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.241641 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.241617    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"]
Nov 03 19:49:16.243599 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.243575    5051 cpu_manager.go:414] "containerMap: removing container" podUID="e44c09d6-0832-49a5-bd7d-f7cae6376d43" containerName="maas-api"
Nov 03 19:49:16.243969 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.243949    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e44c09d6-0832-49a5-bd7d-f7cae6376d43" containerName="maas-api"
Nov 03 19:49:16.244468 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.244452    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="e44c09d6-0832-49a5-bd7d-f7cae6376d43" containerName="maas-api"
Nov 03 19:49:16.249522 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.249503    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.278687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.262635    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-477cd8-openshift-default-dockercfg-vk2t9\""
Nov 03 19:49:16.289182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.289097    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"]
Nov 03 19:49:16.477124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.469285    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082b8718-d755-4a33-9cc5-f1a157b846ec-istiod-ca-cert\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.477124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.469403    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-podinfo\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.477124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.469521    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cvjc\" (UniqueName: \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-kube-api-access-6cvjc\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.477124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.469612    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-token\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.477124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.469677    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.477124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.469745    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-credential-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.477124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.469849    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-data\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.477124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.470012    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-envoy\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.477124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.470086    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-certs\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.575834    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cvjc\" (UniqueName: \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-kube-api-access-6cvjc\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.575937    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-token\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.575998    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.576374    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-credential-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.576475    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-data\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.576616    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-envoy\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.576668    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-certs\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.576796    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082b8718-d755-4a33-9cc5-f1a157b846ec-istiod-ca-cert\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.576828    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-podinfo\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.579425    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-credential-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.584291    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.587264    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-data\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.587573    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-certs\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.593140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.588335    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082b8718-d755-4a33-9cc5-f1a157b846ec-istiod-ca-cert\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.599350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.599322    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-token\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.602953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.602302    5051 generic.go:356] "Generic (PLEG): container finished" podID="ef3fd8a9-7545-4236-86f0-f164991045df" containerID="676ec120c450ea6af22731af7fe1bc5486c1e7094851152feee20da2e30a282c" exitCode=0
Nov 03 19:49:16.602953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.602430    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" event={"ID":"ef3fd8a9-7545-4236-86f0-f164991045df","Type":"ContainerDied","Data":"676ec120c450ea6af22731af7fe1bc5486c1e7094851152feee20da2e30a282c"}
Nov 03 19:49:16.604603 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.604440    5051 generic.go:356] "Generic (PLEG): container finished" podID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerID="36a498222fbedac613e1f161279f33dd21edd6df044cfdca5b3a38ed70d4ea62" exitCode=0
Nov 03 19:49:16.604603 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.604548    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" event={"ID":"6e426eb4-0a92-4fa3-9982-f94cd9a14263","Type":"ContainerDied","Data":"36a498222fbedac613e1f161279f33dd21edd6df044cfdca5b3a38ed70d4ea62"}
Nov 03 19:49:16.606345 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.606212    5051 generic.go:356] "Generic (PLEG): container finished" podID="7d4704e5-7110-4e60-8864-05717066c219" containerID="e1111be2b9ba5609b16c755aecc6ebf6204b78f5552708b95075be7e7d10516b" exitCode=0
Nov 03 19:49:16.606345 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.606287    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" event={"ID":"7d4704e5-7110-4e60-8864-05717066c219","Type":"ContainerDied","Data":"e1111be2b9ba5609b16c755aecc6ebf6204b78f5552708b95075be7e7d10516b"}
Nov 03 19:49:16.620266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.614365    5051 generic.go:356] "Generic (PLEG): container finished" podID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerID="5dfa8d9201e34b7f70a7a97f0e4ef240423af00735575d11aee5a3d143785bb2" exitCode=0
Nov 03 19:49:16.620266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.614461    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" event={"ID":"0efb2272-0d6b-4d8e-b14b-4337b37eba1f","Type":"ContainerDied","Data":"5dfa8d9201e34b7f70a7a97f0e4ef240423af00735575d11aee5a3d143785bb2"}
Nov 03 19:49:16.620266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.614497    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4" event={"ID":"0efb2272-0d6b-4d8e-b14b-4337b37eba1f","Type":"ContainerDied","Data":"6a5dad8a0383dd1fd13608d12424fd41a948db6459ae74eaf749d47b9135de4c"}
Nov 03 19:49:16.620266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.614522    5051 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a5dad8a0383dd1fd13608d12424fd41a948db6459ae74eaf749d47b9135de4c"
Nov 03 19:49:16.620266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.618782    5051 generic.go:356] "Generic (PLEG): container finished" podID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerID="16f478f6fb9ed866513d786dfc713cba63b996b50a99f414dd7e70c9db0a5122" exitCode=0
Nov 03 19:49:16.620266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.618892    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" event={"ID":"97eb9373-4612-4ab1-b511-4b8ea18cb78c","Type":"ContainerDied","Data":"16f478f6fb9ed866513d786dfc713cba63b996b50a99f414dd7e70c9db0a5122"}
Nov 03 19:49:16.622087 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.621825    5051 generic.go:356] "Generic (PLEG): container finished" podID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerID="661af47b265a1d0cb28fbdaf3584e7d1a4c30ef1997e65615060913ef6da2c27" exitCode=0
Nov 03 19:49:16.622087 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.621899    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" event={"ID":"c1ac734c-0c72-4335-9d09-5056d87ceb31","Type":"ContainerDied","Data":"661af47b265a1d0cb28fbdaf3584e7d1a4c30ef1997e65615060913ef6da2c27"}
Nov 03 19:49:16.623718 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.623679    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cvjc\" (UniqueName: \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-kube-api-access-6cvjc\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.624127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.624036    5051 generic.go:356] "Generic (PLEG): container finished" podID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerID="d265ca7b3744977dd923ff9b62d094379a291cfea0cd4318203a814d655116fe" exitCode=0
Nov 03 19:49:16.624253 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.624231    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" event={"ID":"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730","Type":"ContainerDied","Data":"d265ca7b3744977dd923ff9b62d094379a291cfea0cd4318203a814d655116fe"}
Nov 03 19:49:16.627179 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.625895    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-podinfo\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.627179 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.626513    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-envoy\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.630915 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.630891    5051 generic.go:356] "Generic (PLEG): container finished" podID="e44c09d6-0832-49a5-bd7d-f7cae6376d43" containerID="f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c" exitCode=0
Nov 03 19:49:16.631001 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.630925    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm" event={"ID":"e44c09d6-0832-49a5-bd7d-f7cae6376d43","Type":"ContainerDied","Data":"f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c"}
Nov 03 19:49:16.631001 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.630962    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm" event={"ID":"e44c09d6-0832-49a5-bd7d-f7cae6376d43","Type":"ContainerDied","Data":"af8453d030844bae32593b7e59f4fe9b365b0bd1d1dd4b2779c082501708a2e9"}
Nov 03 19:49:16.631001 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.630994    5051 scope.go:122] "RemoveContainer" containerID="f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c"
Nov 03 19:49:16.631399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.631184    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"
Nov 03 19:49:16.718686 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.709866    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:16.747151 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.745728    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"]
Nov 03 19:49:16.748067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.747867    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eae28ff0-6868c8b487-mbwhm"]
Nov 03 19:49:16.768140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.764460    5051 scope.go:122] "RemoveContainer" containerID="f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c"
Nov 03 19:49:16.768140 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:16.765825    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c\": container with ID starting with f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c not found: ID does not exist" containerID="f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c"
Nov 03 19:49:16.768140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.765866    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c"} err="failed to get container status \"f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c\": rpc error: code = NotFound desc = could not find container \"f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c\": container with ID starting with f84dc0555d5eb3b28316a7f44c51c54993546e9b1a9e8f1b50ac07d9e921891c not found: ID does not exist"
Nov 03 19:49:16.774526 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.774489    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:49:16.777261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.777233    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:49:16.839517 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.839093    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:49:16.886051 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.886006    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-token\" (UniqueName: \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-token\") pod \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") "
Nov 03 19:49:16.886196 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.886067    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-certs\") pod \"c1ac734c-0c72-4335-9d09-5056d87ceb31\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") "
Nov 03 19:49:16.886196 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.886121    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-podinfo\") pod \"c1ac734c-0c72-4335-9d09-5056d87ceb31\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") "
Nov 03 19:49:16.886196 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.886160    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istiod-ca-cert\") pod \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") "
Nov 03 19:49:16.886196 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.886187    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-socket\") pod \"ef3fd8a9-7545-4236-86f0-f164991045df\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") "
Nov 03 19:49:16.886326 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.886243    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-kube-api-access-lrvz7\" (UniqueName: \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-kube-api-access-lrvz7\") pod \"ef3fd8a9-7545-4236-86f0-f164991045df\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") "
Nov 03 19:49:16.887096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.886960    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-certs" pod "c1ac734c-0c72-4335-9d09-5056d87ceb31" (UID: "c1ac734c-0c72-4335-9d09-5056d87ceb31"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.887316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.887287    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-socket" pod "ef3fd8a9-7545-4236-86f0-f164991045df" (UID: "ef3fd8a9-7545-4236-86f0-f164991045df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.887464 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.887446    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istiod-ca-cert" pod "0efb2272-0d6b-4d8e-b14b-4337b37eba1f" (UID: "0efb2272-0d6b-4d8e-b14b-4337b37eba1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:16.889470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889437    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-podinfo\") pod \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") "
Nov 03 19:49:16.889566 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889496    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-socket\") pod \"c1ac734c-0c72-4335-9d09-5056d87ceb31\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") "
Nov 03 19:49:16.889566 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889539    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-socket\") pod \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") "
Nov 03 19:49:16.889685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889594    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-data\") pod \"ef3fd8a9-7545-4236-86f0-f164991045df\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") "
Nov 03 19:49:16.889685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889639    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ef3fd8a9-7545-4236-86f0-f164991045df-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef3fd8a9-7545-4236-86f0-f164991045df-istio-podinfo\") pod \"ef3fd8a9-7545-4236-86f0-f164991045df\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") "
Nov 03 19:49:16.889783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889691    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-token\" (UniqueName: \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-token\") pod \"c1ac734c-0c72-4335-9d09-5056d87ceb31\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") "
Nov 03 19:49:16.889783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889750    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef3fd8a9-7545-4236-86f0-f164991045df-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef3fd8a9-7545-4236-86f0-f164991045df-istiod-ca-cert\") pod \"ef3fd8a9-7545-4236-86f0-f164991045df\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") "
Nov 03 19:49:16.889897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889790    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-kube-api-access-mwpvd\" (UniqueName: \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-kube-api-access-mwpvd\") pod \"c1ac734c-0c72-4335-9d09-5056d87ceb31\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") "
Nov 03 19:49:16.889897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889861    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-credential-socket\") pod \"c1ac734c-0c72-4335-9d09-5056d87ceb31\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") "
Nov 03 19:49:16.889897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889886    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-data\") pod \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") "
Nov 03 19:49:16.890089 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889935    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-credential-socket\") pod \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") "
Nov 03 19:49:16.890089 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.889978    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-istio-token\" (UniqueName: \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-istio-token\") pod \"ef3fd8a9-7545-4236-86f0-f164991045df\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") "
Nov 03 19:49:16.890216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890131    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-data\") pod \"c1ac734c-0c72-4335-9d09-5056d87ceb31\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") "
Nov 03 19:49:16.890216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890162    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-credential-socket\") pod \"ef3fd8a9-7545-4236-86f0-f164991045df\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") "
Nov 03 19:49:16.890322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890223    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-envoy\") pod \"c1ac734c-0c72-4335-9d09-5056d87ceb31\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") "
Nov 03 19:49:16.890322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890244    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-socket" pod "0efb2272-0d6b-4d8e-b14b-4337b37eba1f" (UID: "0efb2272-0d6b-4d8e-b14b-4337b37eba1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.890322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890263    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-envoy\") pod \"ef3fd8a9-7545-4236-86f0-f164991045df\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") "
Nov 03 19:49:16.890470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890374    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-certs\") pod \"ef3fd8a9-7545-4236-86f0-f164991045df\" (UID: \"ef3fd8a9-7545-4236-86f0-f164991045df\") "
Nov 03 19:49:16.890470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890431    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-socket" pod "c1ac734c-0c72-4335-9d09-5056d87ceb31" (UID: "c1ac734c-0c72-4335-9d09-5056d87ceb31"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.890586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890526    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1ac734c-0c72-4335-9d09-5056d87ceb31-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1ac734c-0c72-4335-9d09-5056d87ceb31-istiod-ca-cert\") pod \"c1ac734c-0c72-4335-9d09-5056d87ceb31\" (UID: \"c1ac734c-0c72-4335-9d09-5056d87ceb31\") "
Nov 03 19:49:16.890586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890554    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-certs\") pod \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") "
Nov 03 19:49:16.890692 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890622    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-data" pod "ef3fd8a9-7545-4236-86f0-f164991045df" (UID: "ef3fd8a9-7545-4236-86f0-f164991045df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.890751 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890691    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-kube-api-access-n8bc8\" (UniqueName: \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-kube-api-access-n8bc8\") pod \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") "
Nov 03 19:49:16.890751 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890731    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-envoy\") pod \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\" (UID: \"0efb2272-0d6b-4d8e-b14b-4337b37eba1f\") "
Nov 03 19:49:16.890857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.890801    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-data" pod "c1ac734c-0c72-4335-9d09-5056d87ceb31" (UID: "c1ac734c-0c72-4335-9d09-5056d87ceb31"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.891092    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-data" pod "0efb2272-0d6b-4d8e-b14b-4337b37eba1f" (UID: "0efb2272-0d6b-4d8e-b14b-4337b37eba1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.891346    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-credential-socket" pod "0efb2272-0d6b-4d8e-b14b-4337b37eba1f" (UID: "0efb2272-0d6b-4d8e-b14b-4337b37eba1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.891570    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-credential-socket" pod "ef3fd8a9-7545-4236-86f0-f164991045df" (UID: "ef3fd8a9-7545-4236-86f0-f164991045df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.892273    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.892297    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.892312    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.892327    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.892342    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.892356    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.892371    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.892388    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.892417    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.892432    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.893023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.892767    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1ac734c-0c72-4335-9d09-5056d87ceb31-istiod-ca-cert" pod "c1ac734c-0c72-4335-9d09-5056d87ceb31" (UID: "c1ac734c-0c72-4335-9d09-5056d87ceb31"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:16.895737 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.895279    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-envoy" pod "ef3fd8a9-7545-4236-86f0-f164991045df" (UID: "ef3fd8a9-7545-4236-86f0-f164991045df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.895737 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.895504    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-certs" pod "ef3fd8a9-7545-4236-86f0-f164991045df" (UID: "ef3fd8a9-7545-4236-86f0-f164991045df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.903646 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.903466    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef3fd8a9-7545-4236-86f0-f164991045df-istiod-ca-cert" pod "ef3fd8a9-7545-4236-86f0-f164991045df" (UID: "ef3fd8a9-7545-4236-86f0-f164991045df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:16.905506 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.904997    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-credential-socket" pod "c1ac734c-0c72-4335-9d09-5056d87ceb31" (UID: "c1ac734c-0c72-4335-9d09-5056d87ceb31"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.905704 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.905676    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-certs" pod "0efb2272-0d6b-4d8e-b14b-4337b37eba1f" (UID: "0efb2272-0d6b-4d8e-b14b-4337b37eba1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.915576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.909003    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-token" pod "c1ac734c-0c72-4335-9d09-5056d87ceb31" (UID: "c1ac734c-0c72-4335-9d09-5056d87ceb31"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:16.915576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.909087    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-envoy" pod "c1ac734c-0c72-4335-9d09-5056d87ceb31" (UID: "c1ac734c-0c72-4335-9d09-5056d87ceb31"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.915576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.909958    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-envoy" pod "0efb2272-0d6b-4d8e-b14b-4337b37eba1f" (UID: "0efb2272-0d6b-4d8e-b14b-4337b37eba1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:16.915576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.910829    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-kube-api-access-n8bc8" pod "0efb2272-0d6b-4d8e-b14b-4337b37eba1f" (UID: "0efb2272-0d6b-4d8e-b14b-4337b37eba1f"). InnerVolumeSpecName "kube-api-access-n8bc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:16.915576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.913466    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-podinfo" pod "0efb2272-0d6b-4d8e-b14b-4337b37eba1f" (UID: "0efb2272-0d6b-4d8e-b14b-4337b37eba1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:16.915576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.913719    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-kube-api-access-lrvz7" pod "ef3fd8a9-7545-4236-86f0-f164991045df" (UID: "ef3fd8a9-7545-4236-86f0-f164991045df"). InnerVolumeSpecName "kube-api-access-lrvz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:16.915576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.913811    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-podinfo" pod "c1ac734c-0c72-4335-9d09-5056d87ceb31" (UID: "c1ac734c-0c72-4335-9d09-5056d87ceb31"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:16.915576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.913997    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-token" pod "0efb2272-0d6b-4d8e-b14b-4337b37eba1f" (UID: "0efb2272-0d6b-4d8e-b14b-4337b37eba1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:16.915576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.914890    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef3fd8a9-7545-4236-86f0-f164991045df-istio-podinfo" pod "ef3fd8a9-7545-4236-86f0-f164991045df" (UID: "ef3fd8a9-7545-4236-86f0-f164991045df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:16.915576 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.915219    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-kube-api-access-mwpvd" pod "c1ac734c-0c72-4335-9d09-5056d87ceb31" (UID: "c1ac734c-0c72-4335-9d09-5056d87ceb31"). InnerVolumeSpecName "kube-api-access-mwpvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:16.916455 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.916003    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-istio-token" pod "ef3fd8a9-7545-4236-86f0-f164991045df" (UID: "ef3fd8a9-7545-4236-86f0-f164991045df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.998932    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1ac734c-0c72-4335-9d09-5056d87ceb31-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.998964    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.998977    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8bc8\" (UniqueName: \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-kube-api-access-n8bc8\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.998986    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.998998    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999011    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999019    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrvz7\" (UniqueName: \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-kube-api-access-lrvz7\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999029    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0efb2272-0d6b-4d8e-b14b-4337b37eba1f-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999054    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef3fd8a9-7545-4236-86f0-f164991045df-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999067    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999082    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef3fd8a9-7545-4236-86f0-f164991045df-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999094    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwpvd\" (UniqueName: \"kubernetes.io/projected/c1ac734c-0c72-4335-9d09-5056d87ceb31-kube-api-access-mwpvd\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999103    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999111    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef3fd8a9-7545-4236-86f0-f164991045df-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999119    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1ac734c-0c72-4335-9d09-5056d87ceb31-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999127    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:16.999171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:16.999135    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef3fd8a9-7545-4236-86f0-f164991045df-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.055797 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.055542    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:49:17.099893 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.099220    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:49:17.103916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.102093    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-envoy\") pod \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") "
Nov 03 19:49:17.103916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.102219    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-certs\") pod \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") "
Nov 03 19:49:17.103916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.102264    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-kube-api-access-92w9f\" (UniqueName: \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-kube-api-access-92w9f\") pod \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") "
Nov 03 19:49:17.103916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.102517    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-token\" (UniqueName: \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-token\") pod \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") "
Nov 03 19:49:17.103916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.102574    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-socket\") pod \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") "
Nov 03 19:49:17.103916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.102702    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-data\") pod \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") "
Nov 03 19:49:17.103916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.102747    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-podinfo\") pod \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") "
Nov 03 19:49:17.103916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.102828    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istiod-ca-cert\") pod \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") "
Nov 03 19:49:17.103916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.102873    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-credential-socket\") pod \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\" (UID: \"6e426eb4-0a92-4fa3-9982-f94cd9a14263\") "
Nov 03 19:49:17.104496 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.104270    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-certs" pod "6e426eb4-0a92-4fa3-9982-f94cd9a14263" (UID: "6e426eb4-0a92-4fa3-9982-f94cd9a14263"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.110846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.110358    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-data" pod "6e426eb4-0a92-4fa3-9982-f94cd9a14263" (UID: "6e426eb4-0a92-4fa3-9982-f94cd9a14263"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.110846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.110474    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-credential-socket" pod "6e426eb4-0a92-4fa3-9982-f94cd9a14263" (UID: "6e426eb4-0a92-4fa3-9982-f94cd9a14263"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.110846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.110627    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-socket" pod "6e426eb4-0a92-4fa3-9982-f94cd9a14263" (UID: "6e426eb4-0a92-4fa3-9982-f94cd9a14263"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.111108 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.110988    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istiod-ca-cert" pod "6e426eb4-0a92-4fa3-9982-f94cd9a14263" (UID: "6e426eb4-0a92-4fa3-9982-f94cd9a14263"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:17.126156 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.124081    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-token" pod "6e426eb4-0a92-4fa3-9982-f94cd9a14263" (UID: "6e426eb4-0a92-4fa3-9982-f94cd9a14263"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.126156 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.125618    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-envoy" pod "6e426eb4-0a92-4fa3-9982-f94cd9a14263" (UID: "6e426eb4-0a92-4fa3-9982-f94cd9a14263"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.127722 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.127684    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-podinfo" pod "6e426eb4-0a92-4fa3-9982-f94cd9a14263" (UID: "6e426eb4-0a92-4fa3-9982-f94cd9a14263"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:17.127989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.127970    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-kube-api-access-92w9f" pod "6e426eb4-0a92-4fa3-9982-f94cd9a14263" (UID: "6e426eb4-0a92-4fa3-9982-f94cd9a14263"). InnerVolumeSpecName "kube-api-access-92w9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.128285 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.128236    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:49:17.188640 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.188608    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:49:17.205888 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.205853    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-socket\") pod \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") "
Nov 03 19:49:17.206096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.205960    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-podinfo\") pod \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") "
Nov 03 19:49:17.206096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206063    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-data\") pod \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") "
Nov 03 19:49:17.206275 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206108    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-envoy\") pod \"7d4704e5-7110-4e60-8864-05717066c219\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") "
Nov 03 19:49:17.206275 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206136    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-certs\") pod \"7d4704e5-7110-4e60-8864-05717066c219\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") "
Nov 03 19:49:17.206275 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206245    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-envoy\") pod \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") "
Nov 03 19:49:17.206448 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206300    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-credential-socket\") pod \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") "
Nov 03 19:49:17.206448 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206347    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-certs\") pod \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") "
Nov 03 19:49:17.206448 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206382    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7d4704e5-7110-4e60-8864-05717066c219-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d4704e5-7110-4e60-8864-05717066c219-istio-podinfo\") pod \"7d4704e5-7110-4e60-8864-05717066c219\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") "
Nov 03 19:49:17.206608 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206455    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-socket\") pod \"7d4704e5-7110-4e60-8864-05717066c219\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") "
Nov 03 19:49:17.206608 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206486    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-credential-socket\") pod \"7d4704e5-7110-4e60-8864-05717066c219\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") "
Nov 03 19:49:17.206608 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206519    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-data\") pod \"7d4704e5-7110-4e60-8864-05717066c219\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") "
Nov 03 19:49:17.206746 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206606    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-istio-token\" (UniqueName: \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-istio-token\") pod \"7d4704e5-7110-4e60-8864-05717066c219\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") "
Nov 03 19:49:17.206746 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206673    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-kube-api-access-q9fn6\" (UniqueName: \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-kube-api-access-q9fn6\") pod \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") "
Nov 03 19:49:17.206746 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206708    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d4704e5-7110-4e60-8864-05717066c219-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d4704e5-7110-4e60-8864-05717066c219-istiod-ca-cert\") pod \"7d4704e5-7110-4e60-8864-05717066c219\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") "
Nov 03 19:49:17.206859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206749    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istiod-ca-cert\") pod \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") "
Nov 03 19:49:17.206859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206825    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-kube-api-access-jqh6w\" (UniqueName: \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-kube-api-access-jqh6w\") pod \"7d4704e5-7110-4e60-8864-05717066c219\" (UID: \"7d4704e5-7110-4e60-8864-05717066c219\") "
Nov 03 19:49:17.206971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.206951    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-token\" (UniqueName: \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-token\") pod \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\" (UID: \"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730\") "
Nov 03 19:49:17.208030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.208002    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.208149 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.208032    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.208149 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.208061    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.208149 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.208077    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.208149 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.208092    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.208149 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.208106    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.208149 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.208120    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e426eb4-0a92-4fa3-9982-f94cd9a14263-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.208149 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.208134    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92w9f\" (UniqueName: \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-kube-api-access-92w9f\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.208149 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.208148    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e426eb4-0a92-4fa3-9982-f94cd9a14263-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.210989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.210964    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-token" pod "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" (UID: "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.210989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.210978    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d4704e5-7110-4e60-8864-05717066c219-istio-podinfo" pod "7d4704e5-7110-4e60-8864-05717066c219" (UID: "7d4704e5-7110-4e60-8864-05717066c219"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:17.211238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.211217    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-certs" pod "7d4704e5-7110-4e60-8864-05717066c219" (UID: "7d4704e5-7110-4e60-8864-05717066c219"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.211320 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.211232    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-socket" pod "7d4704e5-7110-4e60-8864-05717066c219" (UID: "7d4704e5-7110-4e60-8864-05717066c219"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.211596 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.211574    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-credential-socket" pod "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" (UID: "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.212004 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.211968    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-socket" pod "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" (UID: "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.212151 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.212084    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d4704e5-7110-4e60-8864-05717066c219-istiod-ca-cert" pod "7d4704e5-7110-4e60-8864-05717066c219" (UID: "7d4704e5-7110-4e60-8864-05717066c219"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:17.212238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.212221    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-data" pod "7d4704e5-7110-4e60-8864-05717066c219" (UID: "7d4704e5-7110-4e60-8864-05717066c219"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.212310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.212258    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istiod-ca-cert" pod "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" (UID: "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:17.212425 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.212381    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-certs" pod "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" (UID: "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.212623 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.212449    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-data" pod "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" (UID: "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.216581 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.216553    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-credential-socket" pod "7d4704e5-7110-4e60-8864-05717066c219" (UID: "7d4704e5-7110-4e60-8864-05717066c219"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.217974 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.217391    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-istio-token" pod "7d4704e5-7110-4e60-8864-05717066c219" (UID: "7d4704e5-7110-4e60-8864-05717066c219"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.217974 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.217721    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-envoy" pod "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" (UID: "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.230177 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.230150    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-envoy" pod "7d4704e5-7110-4e60-8864-05717066c219" (UID: "7d4704e5-7110-4e60-8864-05717066c219"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.234375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.234299    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:49:17.240027 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.239642    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-podinfo" pod "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" (UID: "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:17.247084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.245530    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-kube-api-access-q9fn6" pod "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" (UID: "d52e2dd2-1554-47a3-a7c9-5a7a78fe2730"). InnerVolumeSpecName "kube-api-access-q9fn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.268357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.268306    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-kube-api-access-jqh6w" pod "7d4704e5-7110-4e60-8864-05717066c219" (UID: "7d4704e5-7110-4e60-8864-05717066c219"). InnerVolumeSpecName "kube-api-access-jqh6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.271196 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.271164    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e44c09d6-0832-49a5-bd7d-f7cae6376d43" path="/var/lib/kubelet/pods/e44c09d6-0832-49a5-bd7d-f7cae6376d43/volumes"
Nov 03 19:49:17.279031 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.278977    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:49:17.281288 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.281268    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:49:17.285293 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.285272    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:49:17.316289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.316260    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-data\") pod \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") "
Nov 03 19:49:17.316583 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.316537    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-envoy\") pod \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") "
Nov 03 19:49:17.316720 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.316705    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-podinfo\") pod \"ff19db52-e5c7-4f69-aeb3-876bb9488667\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") "
Nov 03 19:49:17.317427 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.317407    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-credential-socket\") pod \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") "
Nov 03 19:49:17.317574 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.317560    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-kube-api-access-zp5fg\" (UniqueName: \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-kube-api-access-zp5fg\") pod \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") "
Nov 03 19:49:17.319365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.319341    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-data\") pod \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") "
Nov 03 19:49:17.319517 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.319501    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-podinfo\") pod \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") "
Nov 03 19:49:17.319635 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.319622    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-credential-socket\") pod \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") "
Nov 03 19:49:17.319833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.319818    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-token\" (UniqueName: \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-token\") pod \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") "
Nov 03 19:49:17.320033 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.320003    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-certs\") pod \"ff19db52-e5c7-4f69-aeb3-876bb9488667\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") "
Nov 03 19:49:17.320182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.320167    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-credential-socket\") pod \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") "
Nov 03 19:49:17.320279 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.320268    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istiod-ca-cert\") pod \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") "
Nov 03 19:49:17.320390 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.320376    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-token\" (UniqueName: \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-token\") pod \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") "
Nov 03 19:49:17.320508 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.320495    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-envoy\") pod \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") "
Nov 03 19:49:17.320659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.320644    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-data\") pod \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") "
Nov 03 19:49:17.321371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.320760    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-kube-api-access-7cx9x\" (UniqueName: \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-kube-api-access-7cx9x\") pod \"ff19db52-e5c7-4f69-aeb3-876bb9488667\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") "
Nov 03 19:49:17.321593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.321573    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9fb4048-87a7-4c19-88f4-c0193b340040-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9fb4048-87a7-4c19-88f4-c0193b340040-istiod-ca-cert\") pod \"e9fb4048-87a7-4c19-88f4-c0193b340040\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") "
Nov 03 19:49:17.321756 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.321734    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-envoy\") pod \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") "
Nov 03 19:49:17.321890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.321876    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-envoy\") pod \"e9fb4048-87a7-4c19-88f4-c0193b340040\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") "
Nov 03 19:49:17.322023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.322009    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-certs\") pod \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") "
Nov 03 19:49:17.322222 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.322205    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-kube-api-access-mhzqn\" (UniqueName: \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-kube-api-access-mhzqn\") pod \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") "
Nov 03 19:49:17.322413 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.322398    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-podinfo\") pod \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") "
Nov 03 19:49:17.331167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.331144    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff19db52-e5c7-4f69-aeb3-876bb9488667-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff19db52-e5c7-4f69-aeb3-876bb9488667-istiod-ca-cert\") pod \"ff19db52-e5c7-4f69-aeb3-876bb9488667\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") "
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.316767    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-data" pod "97eb9373-4612-4ab1-b511-4b8ea18cb78c" (UID: "97eb9373-4612-4ab1-b511-4b8ea18cb78c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.322832    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-credential-socket" pod "f12bfd9b-aae3-489d-a44e-70dab3da12ae" (UID: "f12bfd9b-aae3-489d-a44e-70dab3da12ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.324308    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-data" pod "f12bfd9b-aae3-489d-a44e-70dab3da12ae" (UID: "f12bfd9b-aae3-489d-a44e-70dab3da12ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.324325    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-certs" pod "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" (UID: "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.326215    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9fb4048-87a7-4c19-88f4-c0193b340040-istiod-ca-cert" pod "e9fb4048-87a7-4c19-88f4-c0193b340040" (UID: "e9fb4048-87a7-4c19-88f4-c0193b340040"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.327887    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-credential-socket" pod "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" (UID: "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.328285    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-podinfo" pod "97eb9373-4612-4ab1-b511-4b8ea18cb78c" (UID: "97eb9373-4612-4ab1-b511-4b8ea18cb78c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.328464    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istiod-ca-cert" pod "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" (UID: "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.329065    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-data" pod "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" (UID: "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.330205    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-kube-api-access-mhzqn" pod "97eb9373-4612-4ab1-b511-4b8ea18cb78c" (UID: "97eb9373-4612-4ab1-b511-4b8ea18cb78c"). InnerVolumeSpecName "kube-api-access-mhzqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.330254    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-podinfo" pod "ff19db52-e5c7-4f69-aeb3-876bb9488667" (UID: "ff19db52-e5c7-4f69-aeb3-876bb9488667"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:17.333220 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.330476    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-credential-socket" pod "97eb9373-4612-4ab1-b511-4b8ea18cb78c" (UID: "97eb9373-4612-4ab1-b511-4b8ea18cb78c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.340187 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.340163    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-envoy" pod "e9fb4048-87a7-4c19-88f4-c0193b340040" (UID: "e9fb4048-87a7-4c19-88f4-c0193b340040"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.340815 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.340795    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-kube-api-access-zp5fg" pod "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" (UID: "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c"). InnerVolumeSpecName "kube-api-access-zp5fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.358590 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.358502    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-envoy" pod "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" (UID: "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359257    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-credential-socket\") pod \"ff19db52-e5c7-4f69-aeb3-876bb9488667\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359303    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-certs\") pod \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359334    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-token\" (UniqueName: \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-token\") pod \"e9fb4048-87a7-4c19-88f4-c0193b340040\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359350    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-certs" pod "ff19db52-e5c7-4f69-aeb3-876bb9488667" (UID: "ff19db52-e5c7-4f69-aeb3-876bb9488667"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359370    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-socket\") pod \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359395    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-kube-api-access-l7b2r\" (UniqueName: \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-kube-api-access-l7b2r\") pod \"e9fb4048-87a7-4c19-88f4-c0193b340040\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359426    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-podinfo\") pod \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359453    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-token\" (UniqueName: \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-token\") pod \"ff19db52-e5c7-4f69-aeb3-876bb9488667\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359519    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-socket\") pod \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359534    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-certs" pod "97eb9373-4612-4ab1-b511-4b8ea18cb78c" (UID: "97eb9373-4612-4ab1-b511-4b8ea18cb78c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359561    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-data\") pod \"e9fb4048-87a7-4c19-88f4-c0193b340040\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359629    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-envoy\") pod \"ff19db52-e5c7-4f69-aeb3-876bb9488667\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359668    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-data\") pod \"ff19db52-e5c7-4f69-aeb3-876bb9488667\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359698    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-credential-socket" pod "ff19db52-e5c7-4f69-aeb3-876bb9488667" (UID: "ff19db52-e5c7-4f69-aeb3-876bb9488667"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359698    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-socket\") pod \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359773    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-certs\") pod \"e9fb4048-87a7-4c19-88f4-c0193b340040\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359824    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istiod-ca-cert\") pod \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\" (UID: \"97eb9373-4612-4ab1-b511-4b8ea18cb78c\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359899    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-token\" (UniqueName: \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-token\") pod \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\" (UID: \"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359948    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istiod-ca-cert\") pod \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.359978    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-socket\") pod \"ff19db52-e5c7-4f69-aeb3-876bb9488667\" (UID: \"ff19db52-e5c7-4f69-aeb3-876bb9488667\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.360013    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-kube-api-access-75g4g\" (UniqueName: \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-kube-api-access-75g4g\") pod \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.360068    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-podinfo\") pod \"e9fb4048-87a7-4c19-88f4-c0193b340040\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.360096    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-socket\") pod \"e9fb4048-87a7-4c19-88f4-c0193b340040\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.360110    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff19db52-e5c7-4f69-aeb3-876bb9488667-istiod-ca-cert" pod "ff19db52-e5c7-4f69-aeb3-876bb9488667" (UID: "ff19db52-e5c7-4f69-aeb3-876bb9488667"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.360123    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-credential-socket\") pod \"e9fb4048-87a7-4c19-88f4-c0193b340040\" (UID: \"e9fb4048-87a7-4c19-88f4-c0193b340040\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.360172    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-certs\") pod \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\" (UID: \"f12bfd9b-aae3-489d-a44e-70dab3da12ae\") "
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.360444    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istiod-ca-cert" pod "f12bfd9b-aae3-489d-a44e-70dab3da12ae" (UID: "f12bfd9b-aae3-489d-a44e-70dab3da12ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361273    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361296    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361313    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361328    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9fn6\" (UniqueName: \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-kube-api-access-q9fn6\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361344    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp5fg\" (UniqueName: \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-kube-api-access-zp5fg\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361359    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d4704e5-7110-4e60-8864-05717066c219-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361373    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361388    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361402    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361416    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361430    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqh6w\" (UniqueName: \"kubernetes.io/projected/7d4704e5-7110-4e60-8864-05717066c219-kube-api-access-jqh6w\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361444    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361459    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361473    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361489    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361503    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361517    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361530    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9fb4048-87a7-4c19-88f4-c0193b340040-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361545    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361560    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361573    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361588    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhzqn\" (UniqueName: \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-kube-api-access-mhzqn\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361603    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361618    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff19db52-e5c7-4f69-aeb3-876bb9488667-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361631    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361645    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361659    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361673    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361687    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361701    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361715    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361729    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361744    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d4704e5-7110-4e60-8864-05717066c219-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361758    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361771    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361785    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361800    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.362088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.361814    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d4704e5-7110-4e60-8864-05717066c219-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.369556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.363185    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-socket" pod "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" (UID: "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.369556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.365190    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-certs" pod "f12bfd9b-aae3-489d-a44e-70dab3da12ae" (UID: "f12bfd9b-aae3-489d-a44e-70dab3da12ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.369556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.367112    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-certs" pod "e9fb4048-87a7-4c19-88f4-c0193b340040" (UID: "e9fb4048-87a7-4c19-88f4-c0193b340040"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.369556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.367091    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-socket" pod "ff19db52-e5c7-4f69-aeb3-876bb9488667" (UID: "ff19db52-e5c7-4f69-aeb3-876bb9488667"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.369556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.367761    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-socket" pod "e9fb4048-87a7-4c19-88f4-c0193b340040" (UID: "e9fb4048-87a7-4c19-88f4-c0193b340040"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.369556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.367893    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-credential-socket" pod "e9fb4048-87a7-4c19-88f4-c0193b340040" (UID: "e9fb4048-87a7-4c19-88f4-c0193b340040"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.369556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.368264    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istiod-ca-cert" pod "97eb9373-4612-4ab1-b511-4b8ea18cb78c" (UID: "97eb9373-4612-4ab1-b511-4b8ea18cb78c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:17.369556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.369365    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-socket" pod "f12bfd9b-aae3-489d-a44e-70dab3da12ae" (UID: "f12bfd9b-aae3-489d-a44e-70dab3da12ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.371007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.370979    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-envoy" pod "f12bfd9b-aae3-489d-a44e-70dab3da12ae" (UID: "f12bfd9b-aae3-489d-a44e-70dab3da12ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.376694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.374396    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-data" pod "ff19db52-e5c7-4f69-aeb3-876bb9488667" (UID: "ff19db52-e5c7-4f69-aeb3-876bb9488667"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.376694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.375105    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-data" pod "e9fb4048-87a7-4c19-88f4-c0193b340040" (UID: "e9fb4048-87a7-4c19-88f4-c0193b340040"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.376694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.375161    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-socket" pod "97eb9373-4612-4ab1-b511-4b8ea18cb78c" (UID: "97eb9373-4612-4ab1-b511-4b8ea18cb78c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.378022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.377142    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-podinfo" pod "e9fb4048-87a7-4c19-88f4-c0193b340040" (UID: "e9fb4048-87a7-4c19-88f4-c0193b340040"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:17.384527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.384315    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-podinfo" pod "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" (UID: "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:17.384527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.384442    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-token" pod "ff19db52-e5c7-4f69-aeb3-876bb9488667" (UID: "ff19db52-e5c7-4f69-aeb3-876bb9488667"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.384527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.384478    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-token" pod "97eb9373-4612-4ab1-b511-4b8ea18cb78c" (UID: "97eb9373-4612-4ab1-b511-4b8ea18cb78c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.384527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.384501    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-kube-api-access-7cx9x" pod "ff19db52-e5c7-4f69-aeb3-876bb9488667" (UID: "ff19db52-e5c7-4f69-aeb3-876bb9488667"). InnerVolumeSpecName "kube-api-access-7cx9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.384812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.384583    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-kube-api-access-75g4g" pod "f12bfd9b-aae3-489d-a44e-70dab3da12ae" (UID: "f12bfd9b-aae3-489d-a44e-70dab3da12ae"). InnerVolumeSpecName "kube-api-access-75g4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.384812 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.384584    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-envoy" pod "97eb9373-4612-4ab1-b511-4b8ea18cb78c" (UID: "97eb9373-4612-4ab1-b511-4b8ea18cb78c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.389401 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.389357    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-token" pod "f12bfd9b-aae3-489d-a44e-70dab3da12ae" (UID: "f12bfd9b-aae3-489d-a44e-70dab3da12ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.389768 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.389732    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-podinfo" pod "f12bfd9b-aae3-489d-a44e-70dab3da12ae" (UID: "f12bfd9b-aae3-489d-a44e-70dab3da12ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:17.393793 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.393582    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-kube-api-access-l7b2r" pod "e9fb4048-87a7-4c19-88f4-c0193b340040" (UID: "e9fb4048-87a7-4c19-88f4-c0193b340040"). InnerVolumeSpecName "kube-api-access-l7b2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.394132 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.394099    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-token" pod "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" (UID: "e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.394439 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.394416    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-token" pod "e9fb4048-87a7-4c19-88f4-c0193b340040" (UID: "e9fb4048-87a7-4c19-88f4-c0193b340040"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:17.394649 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.394621    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-envoy" pod "ff19db52-e5c7-4f69-aeb3-876bb9488667" (UID: "ff19db52-e5c7-4f69-aeb3-876bb9488667"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:17.407306 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:17.406266    5051 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d4704e5_7110_4e60_8864_05717066c219.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d4704e5_7110_4e60_8864_05717066c219.slice/crio-654dbcb15e99898de6836f481ad07163ee6ce019be4b1e445542165affd15c32\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef3fd8a9_7545_4236_86f0_f164991045df.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0efb2272_0d6b_4d8e_b14b_4337b37eba1f.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1ac734c_0c72_4335_9d09_5056d87ceb31.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e426eb4_0a92_4fa3_9982_f94cd9a14263.slice/crio-1ae9ac4b138723900490df03d17a7078eb3838fb349c4aa9dc4554e2ea414ea3\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef3fd8a9_7545_4236_86f0_f164991045df.slice/crio-3ca78c2ccdd411943b50104a0d8f66f56114c1731ffbbfd9ce8511010d7a60f2\": RecentStats: unable to find data in memory cache]"
Nov 03 19:49:17.463771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463739    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.463771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463770    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.463771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463785    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75g4g\" (UniqueName: \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-kube-api-access-75g4g\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463801    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463814    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463825    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463837    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463848    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463862    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463874    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463886    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463898    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cx9x\" (UniqueName: \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-kube-api-access-7cx9x\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463912    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463924    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463936    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12bfd9b-aae3-489d-a44e-70dab3da12ae-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463949    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7b2r\" (UniqueName: \"kubernetes.io/projected/e9fb4048-87a7-4c19-88f4-c0193b340040-kube-api-access-l7b2r\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463965    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12bfd9b-aae3-489d-a44e-70dab3da12ae-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463978    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.463993    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97eb9373-4612-4ab1-b511-4b8ea18cb78c-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.464006    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.464019    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.464031    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff19db52-e5c7-4f69-aeb3-876bb9488667-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.464079    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.464097    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9fb4048-87a7-4c19-88f4-c0193b340040-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.464106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.464110    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97eb9373-4612-4ab1-b511-4b8ea18cb78c-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:17.536736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.536683    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"]
Nov 03 19:49:17.540819 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:49:17.540789    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod082b8718_d755_4a33_9cc5_f1a157b846ec.slice/crio-a1e32ef01a7d5879dc52dd5c4b00c2a1483555641eeaa75ca384b70ab21755ae WatchSource:0}: Error finding container a1e32ef01a7d5879dc52dd5c4b00c2a1483555641eeaa75ca384b70ab21755ae: Status 404 returned error can't find the container with id a1e32ef01a7d5879dc52dd5c4b00c2a1483555641eeaa75ca384b70ab21755ae
Nov 03 19:49:17.544487 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.544448    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:17.544590 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.544536    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:17.544590 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.544581    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:17.640863 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.640843    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"
Nov 03 19:49:17.641009 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.640840    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw" event={"ID":"97eb9373-4612-4ab1-b511-4b8ea18cb78c","Type":"ContainerDied","Data":"bbbbcf3fd86448cd85cb3a53b4221da618c1266964c003ca66161f54bf58d207"}
Nov 03 19:49:17.641009 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.640956    5051 scope.go:122] "RemoveContainer" containerID="16f478f6fb9ed866513d786dfc713cba63b996b50a99f414dd7e70c9db0a5122"
Nov 03 19:49:17.642453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.642424    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" event={"ID":"082b8718-d755-4a33-9cc5-f1a157b846ec","Type":"ContainerStarted","Data":"a1e32ef01a7d5879dc52dd5c4b00c2a1483555641eeaa75ca384b70ab21755ae"}
Nov 03 19:49:17.643999 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.643944    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"
Nov 03 19:49:17.644125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.643965    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc" event={"ID":"c1ac734c-0c72-4335-9d09-5056d87ceb31","Type":"ContainerDied","Data":"2f5a8ddf2c25ee7c961137b9f8cc032f066d303486a2fb33515301995fde1c1d"}
Nov 03 19:49:17.645562 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.645541    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"
Nov 03 19:49:17.645640 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.645558    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn" event={"ID":"d52e2dd2-1554-47a3-a7c9-5a7a78fe2730","Type":"ContainerDied","Data":"36f0e0e210f38198d443265347f3820ff0def869686fb9bdbbe1d9c1ff019941"}
Nov 03 19:49:17.649264 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.648657    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn" event={"ID":"ef3fd8a9-7545-4236-86f0-f164991045df","Type":"ContainerDied","Data":"3ca78c2ccdd411943b50104a0d8f66f56114c1731ffbbfd9ce8511010d7a60f2"}
Nov 03 19:49:17.649264 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.648699    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"
Nov 03 19:49:17.650888 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.650863    5051 generic.go:356] "Generic (PLEG): container finished" podID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerID="174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908" exitCode=0
Nov 03 19:49:17.650994 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.650953    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" event={"ID":"e9fb4048-87a7-4c19-88f4-c0193b340040","Type":"ContainerDied","Data":"174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908"}
Nov 03 19:49:17.650994 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.650989    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn" event={"ID":"e9fb4048-87a7-4c19-88f4-c0193b340040","Type":"ContainerDied","Data":"74084e28a43348c42a757843538a82cd3e49876c0cc03bc1633f0170f1d4727a"}
Nov 03 19:49:17.651152 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.651063    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"
Nov 03 19:49:17.653184 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.653159    5051 generic.go:356] "Generic (PLEG): container finished" podID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerID="8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f" exitCode=0
Nov 03 19:49:17.653277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.653253    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" event={"ID":"ff19db52-e5c7-4f69-aeb3-876bb9488667","Type":"ContainerDied","Data":"8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f"}
Nov 03 19:49:17.653332 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.653287    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"
Nov 03 19:49:17.653332 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.653291    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l" event={"ID":"ff19db52-e5c7-4f69-aeb3-876bb9488667","Type":"ContainerDied","Data":"67620e20f243fa30e4681b83d889f0f0269c7fb2de3bfad22c0bedf0e5340030"}
Nov 03 19:49:17.655554 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.654944    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"
Nov 03 19:49:17.655554 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.654971    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9" event={"ID":"6e426eb4-0a92-4fa3-9982-f94cd9a14263","Type":"ContainerDied","Data":"1ae9ac4b138723900490df03d17a7078eb3838fb349c4aa9dc4554e2ea414ea3"}
Nov 03 19:49:17.656378 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.656168    5051 scope.go:122] "RemoveContainer" containerID="661af47b265a1d0cb28fbdaf3584e7d1a4c30ef1997e65615060913ef6da2c27"
Nov 03 19:49:17.657978 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.657924    5051 generic.go:356] "Generic (PLEG): container finished" podID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerID="09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4" exitCode=0
Nov 03 19:49:17.658134 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.658033    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" event={"ID":"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c","Type":"ContainerDied","Data":"09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4"}
Nov 03 19:49:17.658235 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.658152    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp" event={"ID":"e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c","Type":"ContainerDied","Data":"05a8908d24daa1475de700ccd4ec7d74649aa24cb0882c5ea3a958dc14201099"}
Nov 03 19:49:17.658235 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.658110    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"
Nov 03 19:49:17.660151 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.660132    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"
Nov 03 19:49:17.660367 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.660314    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86" event={"ID":"7d4704e5-7110-4e60-8864-05717066c219","Type":"ContainerDied","Data":"654dbcb15e99898de6836f481ad07163ee6ce019be4b1e445542165affd15c32"}
Nov 03 19:49:17.662731 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.662711    5051 generic.go:356] "Generic (PLEG): container finished" podID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerID="3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc" exitCode=0
Nov 03 19:49:17.662896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.662876    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"
Nov 03 19:49:17.663106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.662846    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" event={"ID":"f12bfd9b-aae3-489d-a44e-70dab3da12ae","Type":"ContainerDied","Data":"3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc"}
Nov 03 19:49:17.663106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.663064    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj" event={"ID":"f12bfd9b-aae3-489d-a44e-70dab3da12ae","Type":"ContainerDied","Data":"fee8f7fbf4b38db915fa6cc8ab1b45b65f7785e48abafc7382e748efc0a96e1e"}
Nov 03 19:49:17.663304 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.663291    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"
Nov 03 19:49:17.678386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.678362    5051 scope.go:122] "RemoveContainer" containerID="d265ca7b3744977dd923ff9b62d094379a291cfea0cd4318203a814d655116fe"
Nov 03 19:49:17.711397 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.711368    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"]
Nov 03 19:49:17.720972 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.720945    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-8x6mn"]
Nov 03 19:49:17.727973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.727856    5051 scope.go:122] "RemoveContainer" containerID="676ec120c450ea6af22731af7fe1bc5486c1e7094851152feee20da2e30a282c"
Nov 03 19:49:17.737130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.737102    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"]
Nov 03 19:49:17.750396 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.750366    5051 scope.go:122] "RemoveContainer" containerID="174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908"
Nov 03 19:49:17.751137 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.751084    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-pwzmc"]
Nov 03 19:49:17.755928 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.755910    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"]
Nov 03 19:49:17.761182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.761161    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-pxnsn"]
Nov 03 19:49:17.766176 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.765531    5051 scope.go:122] "RemoveContainer" containerID="174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908"
Nov 03 19:49:17.766318 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:17.766295    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908\": container with ID starting with 174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908 not found: ID does not exist" containerID="174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908"
Nov 03 19:49:17.766409 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.766330    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908"} err="failed to get container status \"174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908\": rpc error: code = NotFound desc = could not find container \"174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908\": container with ID starting with 174aa51a65a8c25c40d9a846b2909a3b604534ea7d553151e0a3cefc63eb2908 not found: ID does not exist"
Nov 03 19:49:17.766409 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.766355    5051 scope.go:122] "RemoveContainer" containerID="8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f"
Nov 03 19:49:17.766530 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.766495    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"]
Nov 03 19:49:17.772572 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.772546    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-n86n4"]
Nov 03 19:49:17.782292 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.782272    5051 scope.go:122] "RemoveContainer" containerID="8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f"
Nov 03 19:49:17.782613 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:17.782590    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f\": container with ID starting with 8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f not found: ID does not exist" containerID="8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f"
Nov 03 19:49:17.782686 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.782624    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f"} err="failed to get container status \"8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f\": rpc error: code = NotFound desc = could not find container \"8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f\": container with ID starting with 8f23ae1730dd95c3aa530e218f96667f3542d66ec3ac986afcf32443d955fb7f not found: ID does not exist"
Nov 03 19:49:17.782686 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.782648    5051 scope.go:122] "RemoveContainer" containerID="36a498222fbedac613e1f161279f33dd21edd6df044cfdca5b3a38ed70d4ea62"
Nov 03 19:49:17.786431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.786384    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"]
Nov 03 19:49:17.790328 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.790302    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-894xw"]
Nov 03 19:49:17.797543 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.797513    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"]
Nov 03 19:49:17.800517 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.800500    5051 scope.go:122] "RemoveContainer" containerID="09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4"
Nov 03 19:49:17.804137 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.804108    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-7rzpp"]
Nov 03 19:49:17.814092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.814070    5051 scope.go:122] "RemoveContainer" containerID="09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4"
Nov 03 19:49:17.814454 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:17.814427    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4\": container with ID starting with 09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4 not found: ID does not exist" containerID="09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4"
Nov 03 19:49:17.814537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.814460    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4"} err="failed to get container status \"09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4\": rpc error: code = NotFound desc = could not find container \"09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4\": container with ID starting with 09bfadb0879a9ae5dc01a668bb4fce9d47d90d20b97a8e9719abd149e937dba4 not found: ID does not exist"
Nov 03 19:49:17.814537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.814484    5051 scope.go:122] "RemoveContainer" containerID="e1111be2b9ba5609b16c755aecc6ebf6204b78f5552708b95075be7e7d10516b"
Nov 03 19:49:17.814537 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.814494    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"]
Nov 03 19:49:17.818959 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.818937    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-n8n86"]
Nov 03 19:49:17.831389 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.831367    5051 scope.go:122] "RemoveContainer" containerID="3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc"
Nov 03 19:49:17.831748 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.831723    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"]
Nov 03 19:49:17.835555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.835530    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-87rpn"]
Nov 03 19:49:17.841807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.841778    5051 scope.go:122] "RemoveContainer" containerID="3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc"
Nov 03 19:49:17.842106 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:17.842086    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc\": container with ID starting with 3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc not found: ID does not exist" containerID="3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc"
Nov 03 19:49:17.842206 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.842113    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc"} err="failed to get container status \"3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc\": rpc error: code = NotFound desc = could not find container \"3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc\": container with ID starting with 3471fc6720174cdc67f8646db53edf9949bed17390d85f122737ccafce5a37bc not found: ID does not exist"
Nov 03 19:49:17.848663 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.848641    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"]
Nov 03 19:49:17.853598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.853571    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-2c3fc4-openshift-default-77ff649d77-x7gt9"]
Nov 03 19:49:17.867566 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.867538    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"]
Nov 03 19:49:17.880104 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.880024    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-sbz8l"]
Nov 03 19:49:17.894015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.893991    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"]
Nov 03 19:49:17.898760 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:17.898728    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-d8a4ea-openshift-default-6b47698d6c-vv2lj"]
Nov 03 19:49:18.679088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:18.679021    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" event={"ID":"082b8718-d755-4a33-9cc5-f1a157b846ec","Type":"ContainerStarted","Data":"eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841"}
Nov 03 19:49:18.710252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:18.710221    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:18.713021 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:18.712964    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" podStartSLOduration=2.71294699 podStartE2EDuration="2.71294699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:18.708062946 +0000 UTC m=+1359.616084903" watchObservedRunningTime="2026-11-03 19:49:18.71294699 +0000 UTC m=+1359.620968930"
Nov 03 19:49:18.716970 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:18.716947    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:19.311152 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.311116    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" path="/var/lib/kubelet/pods/0efb2272-0d6b-4d8e-b14b-4337b37eba1f/volumes"
Nov 03 19:49:19.312703 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.312678    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" path="/var/lib/kubelet/pods/6e426eb4-0a92-4fa3-9982-f94cd9a14263/volumes"
Nov 03 19:49:19.313544 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.313521    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d4704e5-7110-4e60-8864-05717066c219" path="/var/lib/kubelet/pods/7d4704e5-7110-4e60-8864-05717066c219/volumes"
Nov 03 19:49:19.314866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.314847    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" path="/var/lib/kubelet/pods/97eb9373-4612-4ab1-b511-4b8ea18cb78c/volumes"
Nov 03 19:49:19.315646 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.315627    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" path="/var/lib/kubelet/pods/c1ac734c-0c72-4335-9d09-5056d87ceb31/volumes"
Nov 03 19:49:19.316425 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.316407    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" path="/var/lib/kubelet/pods/d52e2dd2-1554-47a3-a7c9-5a7a78fe2730/volumes"
Nov 03 19:49:19.317663 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.317643    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" path="/var/lib/kubelet/pods/e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c/volumes"
Nov 03 19:49:19.318457 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.318425    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" path="/var/lib/kubelet/pods/e9fb4048-87a7-4c19-88f4-c0193b340040/volumes"
Nov 03 19:49:19.319325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.319307    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" path="/var/lib/kubelet/pods/ef3fd8a9-7545-4236-86f0-f164991045df/volumes"
Nov 03 19:49:19.320605 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.320580    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" path="/var/lib/kubelet/pods/f12bfd9b-aae3-489d-a44e-70dab3da12ae/volumes"
Nov 03 19:49:19.321522 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.321504    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" path="/var/lib/kubelet/pods/ff19db52-e5c7-4f69-aeb3-876bb9488667/volumes"
Nov 03 19:49:19.697939 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.697806    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:19.699112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:19.699079    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:49:20.196809 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.196774    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"]
Nov 03 19:49:20.197328 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.197293    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c" podUID="0a90fecd-909e-4eb2-8277-a375d93e3b98" containerName="maas-api" containerID="cri-o://768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89" gracePeriod=30
Nov 03 19:49:20.508809 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.507325    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:49:20.644314 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.644285    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:49:20.711755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.711722    5051 generic.go:356] "Generic (PLEG): container finished" podID="0a90fecd-909e-4eb2-8277-a375d93e3b98" containerID="768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89" exitCode=0
Nov 03 19:49:20.712266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.711791    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"
Nov 03 19:49:20.712266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.711920    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c" event={"ID":"0a90fecd-909e-4eb2-8277-a375d93e3b98","Type":"ContainerDied","Data":"768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89"}
Nov 03 19:49:20.712266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.711969    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c" event={"ID":"0a90fecd-909e-4eb2-8277-a375d93e3b98","Type":"ContainerDied","Data":"db6566009b17d778533e0379da3ef8c3783a8c0bd0250e2b55fb86f99ea4521a"}
Nov 03 19:49:20.712266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.712003    5051 scope.go:122] "RemoveContainer" containerID="768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89"
Nov 03 19:49:20.726771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.726753    5051 scope.go:122] "RemoveContainer" containerID="768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89"
Nov 03 19:49:20.727163 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:20.727142    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89\": container with ID starting with 768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89 not found: ID does not exist" containerID="768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89"
Nov 03 19:49:20.727259 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.727218    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89"} err="failed to get container status \"768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89\": rpc error: code = NotFound desc = could not find container \"768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89\": container with ID starting with 768ea3c45238a9cb7aecd1071a431d5df64537cb261cf4b1866c816dc11ecb89 not found: ID does not exist"
Nov 03 19:49:20.740600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.740572    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a90fecd-909e-4eb2-8277-a375d93e3b98-kube-api-access-pt9t2\" (UniqueName: \"kubernetes.io/projected/0a90fecd-909e-4eb2-8277-a375d93e3b98-kube-api-access-pt9t2\") pod \"0a90fecd-909e-4eb2-8277-a375d93e3b98\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") "
Nov 03 19:49:20.740736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.740715    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-maas-api-tls\") pod \"0a90fecd-909e-4eb2-8277-a375d93e3b98\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") "
Nov 03 19:49:20.740816 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.740760    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a90fecd-909e-4eb2-8277-a375d93e3b98-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a90fecd-909e-4eb2-8277-a375d93e3b98-openshift-service-ca\") pod \"0a90fecd-909e-4eb2-8277-a375d93e3b98\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") "
Nov 03 19:49:20.740886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.740814    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-metrics-certs\") pod \"0a90fecd-909e-4eb2-8277-a375d93e3b98\" (UID: \"0a90fecd-909e-4eb2-8277-a375d93e3b98\") "
Nov 03 19:49:20.741736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.741705    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a90fecd-909e-4eb2-8277-a375d93e3b98-openshift-service-ca" pod "0a90fecd-909e-4eb2-8277-a375d93e3b98" (UID: "0a90fecd-909e-4eb2-8277-a375d93e3b98"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:20.742675 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.742649    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a90fecd-909e-4eb2-8277-a375d93e3b98-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:20.748326 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.748302    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-metrics-certs" pod "0a90fecd-909e-4eb2-8277-a375d93e3b98" (UID: "0a90fecd-909e-4eb2-8277-a375d93e3b98"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:49:20.748407 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.748343    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a90fecd-909e-4eb2-8277-a375d93e3b98-kube-api-access-pt9t2" pod "0a90fecd-909e-4eb2-8277-a375d93e3b98" (UID: "0a90fecd-909e-4eb2-8277-a375d93e3b98"). InnerVolumeSpecName "kube-api-access-pt9t2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:20.749491 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.749466    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-maas-api-tls" pod "0a90fecd-909e-4eb2-8277-a375d93e3b98" (UID: "0a90fecd-909e-4eb2-8277-a375d93e3b98"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:49:20.844786 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.844699    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt9t2\" (UniqueName: \"kubernetes.io/projected/0a90fecd-909e-4eb2-8277-a375d93e3b98-kube-api-access-pt9t2\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:20.844786 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.844741    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:20.844786 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:20.844754    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a90fecd-909e-4eb2-8277-a375d93e3b98-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:21.058609 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:21.058568    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"]
Nov 03 19:49:21.064926 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:21.064897    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f292e03c-5b8cf97f6b-ml25c"]
Nov 03 19:49:21.260875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:21.256763    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a90fecd-909e-4eb2-8277-a375d93e3b98" path="/var/lib/kubelet/pods/0a90fecd-909e-4eb2-8277-a375d93e3b98/volumes"
Nov 03 19:49:22.448859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.448830    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"]
Nov 03 19:49:22.449527 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449512    5051 cpu_manager.go:414] "containerMap: removing container" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy"
Nov 03 19:49:22.449586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449528    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy"
Nov 03 19:49:22.449586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449546    5051 cpu_manager.go:414] "containerMap: removing container" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy"
Nov 03 19:49:22.449586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449552    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy"
Nov 03 19:49:22.449586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449562    5051 cpu_manager.go:414] "containerMap: removing container" podUID="0a90fecd-909e-4eb2-8277-a375d93e3b98" containerName="maas-api"
Nov 03 19:49:22.449586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449569    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a90fecd-909e-4eb2-8277-a375d93e3b98" containerName="maas-api"
Nov 03 19:49:22.449586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449583    5051 cpu_manager.go:414] "containerMap: removing container" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449588    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449619    5051 cpu_manager.go:414] "containerMap: removing container" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449624    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449633    5051 cpu_manager.go:414] "containerMap: removing container" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449638    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449645    5051 cpu_manager.go:414] "containerMap: removing container" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449649    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449662    5051 cpu_manager.go:414] "containerMap: removing container" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449667    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449676    5051 cpu_manager.go:414] "containerMap: removing container" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449681    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449688    5051 cpu_manager.go:414] "containerMap: removing container" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449693    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449705    5051 cpu_manager.go:414] "containerMap: removing container" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449710    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449716    5051 cpu_manager.go:414] "containerMap: removing container" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy"
Nov 03 19:49:22.449750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449720    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449814    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0ee4b15-e7d5-4da4-b1bd-9d86c9b80d7c" containerName="istio-proxy"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449822    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d4704e5-7110-4e60-8864-05717066c219" containerName="istio-proxy"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449828    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="0efb2272-0d6b-4d8e-b14b-4337b37eba1f" containerName="istio-proxy"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449837    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="97eb9373-4612-4ab1-b511-4b8ea18cb78c" containerName="istio-proxy"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449843    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a90fecd-909e-4eb2-8277-a375d93e3b98" containerName="maas-api"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449849    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef3fd8a9-7545-4236-86f0-f164991045df" containerName="istio-proxy"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449860    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9fb4048-87a7-4c19-88f4-c0193b340040" containerName="istio-proxy"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449867    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="f12bfd9b-aae3-489d-a44e-70dab3da12ae" containerName="istio-proxy"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449874    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff19db52-e5c7-4f69-aeb3-876bb9488667" containerName="istio-proxy"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449889    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="d52e2dd2-1554-47a3-a7c9-5a7a78fe2730" containerName="istio-proxy"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449896    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1ac734c-0c72-4335-9d09-5056d87ceb31" containerName="istio-proxy"
Nov 03 19:49:22.450258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.449903    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e426eb4-0a92-4fa3-9982-f94cd9a14263" containerName="istio-proxy"
Nov 03 19:49:22.455077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.455019    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:49:22.464392 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.464340    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-477cd8-dockercfg-28h28\""
Nov 03 19:49:22.464798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.464775    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"]
Nov 03 19:49:22.469543 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.469524    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:49:22.469690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.469666    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"]
Nov 03 19:49:22.479470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.479445    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaab2337-0cb8-4c1b-8155-45ca5ebad934-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv\" (UID: \"eaab2337-0cb8-4c1b-8155-45ca5ebad934\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:49:22.479672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.479651    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl2sv\" (UniqueName: \"kubernetes.io/projected/f4ebb6dd-147d-4669-b280-2f77e334f18b-kube-api-access-cl2sv\") pod \"payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh\" (UID: \"f4ebb6dd-147d-4669-b280-2f77e334f18b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:49:22.479987 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.479965    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhxxq\" (UniqueName: \"kubernetes.io/projected/eaab2337-0cb8-4c1b-8155-45ca5ebad934-kube-api-access-qhxxq\") pod \"payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv\" (UID: \"eaab2337-0cb8-4c1b-8155-45ca5ebad934\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:49:22.480209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.480193    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ebb6dd-147d-4669-b280-2f77e334f18b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh\" (UID: \"f4ebb6dd-147d-4669-b280-2f77e334f18b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:49:22.485582 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.484825    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"]
Nov 03 19:49:22.488643 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.488618    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-477cd8\""
Nov 03 19:49:22.602066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.599671    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaab2337-0cb8-4c1b-8155-45ca5ebad934-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv\" (UID: \"eaab2337-0cb8-4c1b-8155-45ca5ebad934\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:49:22.602066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.599769    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl2sv\" (UniqueName: \"kubernetes.io/projected/f4ebb6dd-147d-4669-b280-2f77e334f18b-kube-api-access-cl2sv\") pod \"payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh\" (UID: \"f4ebb6dd-147d-4669-b280-2f77e334f18b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:49:22.602066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.599854    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhxxq\" (UniqueName: \"kubernetes.io/projected/eaab2337-0cb8-4c1b-8155-45ca5ebad934-kube-api-access-qhxxq\") pod \"payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv\" (UID: \"eaab2337-0cb8-4c1b-8155-45ca5ebad934\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:49:22.602066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.599896    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ebb6dd-147d-4669-b280-2f77e334f18b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh\" (UID: \"f4ebb6dd-147d-4669-b280-2f77e334f18b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:49:22.602066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.600980    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ebb6dd-147d-4669-b280-2f77e334f18b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh\" (UID: \"f4ebb6dd-147d-4669-b280-2f77e334f18b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:49:22.602066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.601728    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaab2337-0cb8-4c1b-8155-45ca5ebad934-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv\" (UID: \"eaab2337-0cb8-4c1b-8155-45ca5ebad934\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:49:22.710115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.709939    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl2sv\" (UniqueName: \"kubernetes.io/projected/f4ebb6dd-147d-4669-b280-2f77e334f18b-kube-api-access-cl2sv\") pod \"payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh\" (UID: \"f4ebb6dd-147d-4669-b280-2f77e334f18b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:49:22.718129 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.713256    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhxxq\" (UniqueName: \"kubernetes.io/projected/eaab2337-0cb8-4c1b-8155-45ca5ebad934-kube-api-access-qhxxq\") pod \"payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv\" (UID: \"eaab2337-0cb8-4c1b-8155-45ca5ebad934\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:49:22.779402 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.779368    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:49:22.793902 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:22.793870    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:49:23.090859 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:49:23.090833    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaab2337_0cb8_4c1b_8155_45ca5ebad934.slice/crio-19d0da0f6a6bddfc329feb7b615777c5ab644234464f083c137651b70590e01d WatchSource:0}: Error finding container 19d0da0f6a6bddfc329feb7b615777c5ab644234464f083c137651b70590e01d: Status 404 returned error can't find the container with id 19d0da0f6a6bddfc329feb7b615777c5ab644234464f083c137651b70590e01d
Nov 03 19:49:23.091241 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:23.091218    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"]
Nov 03 19:49:23.120763 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:49:23.120738    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ebb6dd_147d_4669_b280_2f77e334f18b.slice/crio-409f29a5efbce68b0c337c9370d92986defff4c46a2bf79939d8d775dc165a0b WatchSource:0}: Error finding container 409f29a5efbce68b0c337c9370d92986defff4c46a2bf79939d8d775dc165a0b: Status 404 returned error can't find the container with id 409f29a5efbce68b0c337c9370d92986defff4c46a2bf79939d8d775dc165a0b
Nov 03 19:49:23.121922 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:23.121898    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"]
Nov 03 19:49:23.746683 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:23.746644    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv" event={"ID":"eaab2337-0cb8-4c1b-8155-45ca5ebad934","Type":"ContainerStarted","Data":"82a62337483b2c1c89d7004ff499e635dfcdcb821d61d9fd186f60458ac6ff61"}
Nov 03 19:49:23.747152 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:23.746715    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:49:23.747152 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:23.746742    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv" event={"ID":"eaab2337-0cb8-4c1b-8155-45ca5ebad934","Type":"ContainerStarted","Data":"19d0da0f6a6bddfc329feb7b615777c5ab644234464f083c137651b70590e01d"}
Nov 03 19:49:23.748426 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:23.748399    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh" event={"ID":"f4ebb6dd-147d-4669-b280-2f77e334f18b","Type":"ContainerStarted","Data":"58b19ff9f6f396aa154ed3fa3c6410dc3fc7409fae83ab2a622b122ac6241ba2"}
Nov 03 19:49:23.748534 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:23.748451    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh" event={"ID":"f4ebb6dd-147d-4669-b280-2f77e334f18b","Type":"ContainerStarted","Data":"409f29a5efbce68b0c337c9370d92986defff4c46a2bf79939d8d775dc165a0b"}
Nov 03 19:49:23.748534 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:23.748479    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:49:23.766609 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:23.766562    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv" podStartSLOduration=1.766550687 podStartE2EDuration="1.766550687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:23.765947124 +0000 UTC m=+1364.673969070" watchObservedRunningTime="2026-11-03 19:49:23.766550687 +0000 UTC m=+1364.674572626"
Nov 03 19:49:23.788673 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:23.788633    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh" podStartSLOduration=1.788622294 podStartE2EDuration="1.788622294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:23.784377038 +0000 UTC m=+1364.692398979" watchObservedRunningTime="2026-11-03 19:49:23.788622294 +0000 UTC m=+1364.696644232"
Nov 03 19:49:25.786297 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:25.786263    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"]
Nov 03 19:49:25.786665 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:25.786466    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerName="istio-proxy" containerID="cri-o://5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18" gracePeriod=30
Nov 03 19:49:30.695241 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.695202    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"]
Nov 03 19:49:30.700760 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.700727    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:30.711633 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.709227    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-300831-openshift-default-dockercfg-xgk46\""
Nov 03 19:49:30.718450 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.718424    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"]
Nov 03 19:49:30.894072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.893397    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-data\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:30.894072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.893444    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-credential-socket\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:30.894072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.893526    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-token\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:30.894072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.893573    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-socket\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:30.894072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.893729    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-certs\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:30.894072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.893773    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cckl7\" (UniqueName: \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-kube-api-access-cckl7\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:30.894072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.893811    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/425fcd7d-1420-41a8-89d4-503a9d259b89-istiod-ca-cert\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:30.894072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.893852    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-podinfo\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:30.894072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.893902    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-envoy\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:30.995518 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.995479    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-data\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.999284    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-credential-socket\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.999380    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-token\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.999421    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-socket\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.999545    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-certs\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.999577    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cckl7\" (UniqueName: \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-kube-api-access-cckl7\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.999602    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/425fcd7d-1420-41a8-89d4-503a9d259b89-istiod-ca-cert\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.999633    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-podinfo\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.999665    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-envoy\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.000407    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-credential-socket\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:30.997682    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-data\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.001155    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/425fcd7d-1420-41a8-89d4-503a9d259b89-istiod-ca-cert\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.001588    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-socket\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.002221    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-certs\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.004163    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-envoy\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.005932 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.005135    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-podinfo\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.023696 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.023667    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-token\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.025317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.025289    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cckl7\" (UniqueName: \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-kube-api-access-cckl7\") pod \"e2e-api-a-300831-openshift-default-656d95d487-xtm2g\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") " pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.186676 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.186648    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:49:31.203660 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.203630    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-envoy\") pod \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") "
Nov 03 19:49:31.203815 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.203729    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-token\" (UniqueName: \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-token\") pod \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") "
Nov 03 19:49:31.203815 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.203770    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-credential-socket\") pod \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") "
Nov 03 19:49:31.203891 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.203833    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-certs\") pod \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") "
Nov 03 19:49:31.203891 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.203870    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-socket\") pod \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") "
Nov 03 19:49:31.203997 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.203930    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-podinfo\") pod \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") "
Nov 03 19:49:31.203997 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.203967    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istiod-ca-cert\") pod \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") "
Nov 03 19:49:31.204128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.204026    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-kube-api-access-grf5k\" (UniqueName: \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-kube-api-access-grf5k\") pod \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") "
Nov 03 19:49:31.204128 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.204097    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-data\") pod \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\" (UID: \"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3\") "
Nov 03 19:49:31.204228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.204187    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-credential-socket" pod "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" (UID: "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:31.204228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.204199    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-certs" pod "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" (UID: "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:31.204228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.204211    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-socket" pod "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" (UID: "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:31.204445 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.204402    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-data" pod "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" (UID: "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:31.204575 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.204551    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istiod-ca-cert" pod "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" (UID: "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:49:31.205182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.205159    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:31.205298 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.205186    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:31.205298 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.205201    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:31.205298 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.205215    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:31.205298 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.205229    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:31.206240 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.206213    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-envoy" pod "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" (UID: "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:49:31.206549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.206529    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-podinfo" pod "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" (UID: "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:49:31.206549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.206544    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-kube-api-access-grf5k" pod "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" (UID: "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3"). InnerVolumeSpecName "kube-api-access-grf5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:31.206681 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.206656    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-token" pod "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" (UID: "d0886ebc-5f57-4b08-b946-a7bb50bcbdd3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:49:31.309594 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.308100    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:31.309594 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.308136    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grf5k\" (UniqueName: \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-kube-api-access-grf5k\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:31.309594 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.308155    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:31.309594 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.308169    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:49:31.323597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.323564    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:31.631256 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:49:31.631203    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod425fcd7d_1420_41a8_89d4_503a9d259b89.slice/crio-4ef46fa22f72620a7ece283768366b93f791802c5e0b8c990e45d044077ae957 WatchSource:0}: Error finding container 4ef46fa22f72620a7ece283768366b93f791802c5e0b8c990e45d044077ae957: Status 404 returned error can't find the container with id 4ef46fa22f72620a7ece283768366b93f791802c5e0b8c990e45d044077ae957
Nov 03 19:49:31.635844 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.635819    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"]
Nov 03 19:49:31.637011 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.636959    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:31.637136 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.637114    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:31.637281 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.637206    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:31.909900 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.909804    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" event={"ID":"425fcd7d-1420-41a8-89d4-503a9d259b89","Type":"ContainerStarted","Data":"96d8f4c3450d20f32bb5000ef953f7fd9e9e66ec16cb4849373169f4faebf089"}
Nov 03 19:49:31.909900 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.909870    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" event={"ID":"425fcd7d-1420-41a8-89d4-503a9d259b89","Type":"ContainerStarted","Data":"4ef46fa22f72620a7ece283768366b93f791802c5e0b8c990e45d044077ae957"}
Nov 03 19:49:31.912303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.912278    5051 generic.go:356] "Generic (PLEG): container finished" podID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerID="5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18" exitCode=0
Nov 03 19:49:31.912414 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.912352    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"
Nov 03 19:49:31.912414 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.912367    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" event={"ID":"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3","Type":"ContainerDied","Data":"5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18"}
Nov 03 19:49:31.912513 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.912429    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m" event={"ID":"d0886ebc-5f57-4b08-b946-a7bb50bcbdd3","Type":"ContainerDied","Data":"146dce3251a25f71ce39aab321fc92cba01c2482b9bba3807e34c9142bc9b39a"}
Nov 03 19:49:31.912513 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.912462    5051 scope.go:122] "RemoveContainer" containerID="5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18"
Nov 03 19:49:31.924919 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.924899    5051 scope.go:122] "RemoveContainer" containerID="5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18"
Nov 03 19:49:31.925269 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:31.925246    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18\": container with ID starting with 5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18 not found: ID does not exist" containerID="5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18"
Nov 03 19:49:31.925361 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.925275    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18"} err="failed to get container status \"5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18\": rpc error: code = NotFound desc = could not find container \"5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18\": container with ID starting with 5c6de9006783271d8d915e939a519bcf893e3d2d02d2d90438805f07e4bc4f18 not found: ID does not exist"
Nov 03 19:49:31.935058 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.934997    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" podStartSLOduration=1.934982006 podStartE2EDuration="1.934982006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:31.933071115 +0000 UTC m=+1372.841093059" watchObservedRunningTime="2026-11-03 19:49:31.934982006 +0000 UTC m=+1372.843003949"
Nov 03 19:49:31.952553 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.952526    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"]
Nov 03 19:49:31.958849 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:31.958825    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-7vp9m"]
Nov 03 19:49:32.324339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.324308    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:32.330213 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.330185    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:32.368533 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.368505    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"]
Nov 03 19:49:32.369442 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.369416    5051 cpu_manager.go:414] "containerMap: removing container" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerName="istio-proxy"
Nov 03 19:49:32.369442 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.369436    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerName="istio-proxy"
Nov 03 19:49:32.369652 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.369639    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" containerName="istio-proxy"
Nov 03 19:49:32.374581 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.374560    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.390217 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.390194    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-8cff38-openshift-default-dockercfg-6v4sz\""
Nov 03 19:49:32.411999 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.411971    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"]
Nov 03 19:49:32.436120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.435970    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istiod-ca-cert\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.436120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.436017    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-token\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.436120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.436064    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-podinfo\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.436525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.436149    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-socket\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.436525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.436265    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-data\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.436525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.436350    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wffl5\" (UniqueName: \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-kube-api-access-wffl5\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.436525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.436406    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-certs\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.436525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.436513    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-credential-socket\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.436921 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.436601    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-envoy\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.555647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.555609    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-token\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.555946 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.555929    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-podinfo\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.556130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.556113    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-socket\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.556322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.556307    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-data\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.556472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.556457    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wffl5\" (UniqueName: \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-kube-api-access-wffl5\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.556601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.556588    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-certs\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.556762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.556750    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-credential-socket\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.556906 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.556892    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-envoy\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.557031 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.557018    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istiod-ca-cert\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.557530 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.557483    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-data\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.558169 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.558138    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istiod-ca-cert\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.558761 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.558740    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-certs\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.559109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.559090    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-credential-socket\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.564935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.562754    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-socket\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.574194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.570490    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-token\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.574194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.573315    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wffl5\" (UniqueName: \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-kube-api-access-wffl5\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.574710 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.574662    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-envoy\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.575625 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.575603    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-podinfo\") pod \"e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") " pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.691517 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.691485    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:32.782082 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:32.781831    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:49:32.922024 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.921948    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:32.923027 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.923001    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:49:32.997086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.988647    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"]
Nov 03 19:49:32.997086 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:49:32.991019    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0a399e_8e57_4d85_bdb5_94577f83a19c.slice/crio-bd3504c89b414f1a5ced873466e08566b3308fd10c3d20262fc23eb4af8a5698 WatchSource:0}: Error finding container bd3504c89b414f1a5ced873466e08566b3308fd10c3d20262fc23eb4af8a5698: Status 404 returned error can't find the container with id bd3504c89b414f1a5ced873466e08566b3308fd10c3d20262fc23eb4af8a5698
Nov 03 19:49:32.997086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.994649    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:32.997086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.994754    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:32.997086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:32.994803    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:33.232318 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:33.232283    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0886ebc-5f57-4b08-b946-a7bb50bcbdd3" path="/var/lib/kubelet/pods/d0886ebc-5f57-4b08-b946-a7bb50bcbdd3/volumes"
Nov 03 19:49:33.932350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:33.932308    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" event={"ID":"4b0a399e-8e57-4d85-bdb5-94577f83a19c","Type":"ContainerStarted","Data":"d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4"}
Nov 03 19:49:33.932704 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:33.932375    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" event={"ID":"4b0a399e-8e57-4d85-bdb5-94577f83a19c","Type":"ContainerStarted","Data":"bd3504c89b414f1a5ced873466e08566b3308fd10c3d20262fc23eb4af8a5698"}
Nov 03 19:49:33.961770 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:33.961718    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" podStartSLOduration=1.9617055890000001 podStartE2EDuration="1.961705589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:33.958745051 +0000 UTC m=+1374.866767022" watchObservedRunningTime="2026-11-03 19:49:33.961705589 +0000 UTC m=+1374.869727527"
Nov 03 19:49:34.251418 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.251387    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"]
Nov 03 19:49:34.256346 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.256322    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.266787 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.266744    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-477cd8-serving-cert\""
Nov 03 19:49:34.273916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.273887    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"]
Nov 03 19:49:34.418317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.413899    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.423805 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.423743    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.424125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.424097    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rlhr\" (UniqueName: \"kubernetes.io/projected/30d163b8-cbff-40ac-85a7-5f8438d82d4a-kube-api-access-5rlhr\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.424225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.424154    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30d163b8-cbff-40ac-85a7-5f8438d82d4a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.525709 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.525623    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.525851 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.525742    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rlhr\" (UniqueName: \"kubernetes.io/projected/30d163b8-cbff-40ac-85a7-5f8438d82d4a-kube-api-access-5rlhr\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.525851 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.525765    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30d163b8-cbff-40ac-85a7-5f8438d82d4a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.525851 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.525805    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.526547 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.526525    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30d163b8-cbff-40ac-85a7-5f8438d82d4a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.528695 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.528658    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.529221 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.529199    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.538225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.538200    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rlhr\" (UniqueName: \"kubernetes.io/projected/30d163b8-cbff-40ac-85a7-5f8438d82d4a-kube-api-access-5rlhr\") pod \"maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.577292 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.577268    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:34.691861 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.691823    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:34.698986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.698955    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:34.758455 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.758430    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:49:34.758705 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.758690    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:49:34.872843 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:49:34.872800    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30d163b8_cbff_40ac_85a7_5f8438d82d4a.slice/crio-b0445619d97b38ea859a2a14ccbbb4780155272ddb7f2a34440df670dca45e24 WatchSource:0}: Error finding container b0445619d97b38ea859a2a14ccbbb4780155272ddb7f2a34440df670dca45e24: Status 404 returned error can't find the container with id b0445619d97b38ea859a2a14ccbbb4780155272ddb7f2a34440df670dca45e24
Nov 03 19:49:34.877086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:34.877061    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"]
Nov 03 19:49:35.029243 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:35.028857    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9" event={"ID":"30d163b8-cbff-40ac-85a7-5f8438d82d4a","Type":"ContainerStarted","Data":"b0445619d97b38ea859a2a14ccbbb4780155272ddb7f2a34440df670dca45e24"}
Nov 03 19:49:35.030157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:35.030129    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:35.080061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:35.078371    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:49:36.041846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.041308    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9" event={"ID":"30d163b8-cbff-40ac-85a7-5f8438d82d4a","Type":"ContainerStarted","Data":"b7ddcee88e52d50cfc1cef5b562ffa7993198435a8a4814f0ddf22f723631baa"}
Nov 03 19:49:36.041846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.041384    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:36.072171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.068105    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9" podStartSLOduration=1.572273223 podStartE2EDuration="2.06803355s" totalImagesPullingTime="495.760327ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:36.066225603 +0000 UTC m=+1376.974247557" watchObservedRunningTime="2026-11-03 19:49:36.06803355 +0000 UTC m=+1376.976055494"
Nov 03 19:49:36.208010 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.207974    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"]
Nov 03 19:49:36.215411 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.215370    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:49:36.226093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.223325    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-300831\""
Nov 03 19:49:36.226093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.224199    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-300831-dockercfg-wqkxj\""
Nov 03 19:49:36.247643 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.247610    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"]
Nov 03 19:49:36.254858 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.254802    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"]
Nov 03 19:49:36.264558 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.264528    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:49:36.265011 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.264987    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"]
Nov 03 19:49:36.310799 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.306685    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a1a4117-0b43-4d57-8023-25bed714d499-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm\" (UID: \"9a1a4117-0b43-4d57-8023-25bed714d499\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:49:36.310799 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.306843    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-798hg\" (UniqueName: \"kubernetes.io/projected/9a1a4117-0b43-4d57-8023-25bed714d499-kube-api-access-798hg\") pod \"payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm\" (UID: \"9a1a4117-0b43-4d57-8023-25bed714d499\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:49:36.592214 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.592132    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljkgh\" (UniqueName: \"kubernetes.io/projected/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-kube-api-access-ljkgh\") pod \"payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr\" (UID: \"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d\") " pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:49:36.592349 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.592294    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a1a4117-0b43-4d57-8023-25bed714d499-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm\" (UID: \"9a1a4117-0b43-4d57-8023-25bed714d499\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:49:36.592349 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.592322    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr\" (UID: \"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d\") " pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:49:36.592431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.592404    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-798hg\" (UniqueName: \"kubernetes.io/projected/9a1a4117-0b43-4d57-8023-25bed714d499-kube-api-access-798hg\") pod \"payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm\" (UID: \"9a1a4117-0b43-4d57-8023-25bed714d499\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:49:36.593006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.592986    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a1a4117-0b43-4d57-8023-25bed714d499-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm\" (UID: \"9a1a4117-0b43-4d57-8023-25bed714d499\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:49:36.605231 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.605198    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-798hg\" (UniqueName: \"kubernetes.io/projected/9a1a4117-0b43-4d57-8023-25bed714d499-kube-api-access-798hg\") pod \"payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm\" (UID: \"9a1a4117-0b43-4d57-8023-25bed714d499\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:49:36.693595 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.693567    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr\" (UID: \"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d\") " pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:49:36.693856 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.693834    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljkgh\" (UniqueName: \"kubernetes.io/projected/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-kube-api-access-ljkgh\") pod \"payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr\" (UID: \"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d\") " pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:49:36.694364 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.694346    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr\" (UID: \"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d\") " pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:49:36.734166 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.734098    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljkgh\" (UniqueName: \"kubernetes.io/projected/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-kube-api-access-ljkgh\") pod \"payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr\" (UID: \"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d\") " pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:49:36.857035 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:36.856960    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:49:37.013651 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:37.013621    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:49:37.146307 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:49:37.146271    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a1a4117_0b43_4d57_8023_25bed714d499.slice/crio-5a091c4bf8eced8dead2e1010bf4ece53d7386b52b4b551af70da3005c35f93a WatchSource:0}: Error finding container 5a091c4bf8eced8dead2e1010bf4ece53d7386b52b4b551af70da3005c35f93a: Status 404 returned error can't find the container with id 5a091c4bf8eced8dead2e1010bf4ece53d7386b52b4b551af70da3005c35f93a
Nov 03 19:49:37.149185 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:37.149153    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"]
Nov 03 19:49:37.316460 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:37.316432    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"]
Nov 03 19:49:37.316678 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:49:37.316661    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0336ae_47dd_45db_9a74_1aaf1bbcb47d.slice/crio-d842035b88d2013f07988af9b8f8fc4aec93cb7d98325efae64b83e543381b5f WatchSource:0}: Error finding container d842035b88d2013f07988af9b8f8fc4aec93cb7d98325efae64b83e543381b5f: Status 404 returned error can't find the container with id d842035b88d2013f07988af9b8f8fc4aec93cb7d98325efae64b83e543381b5f
Nov 03 19:49:38.061867 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:38.061824    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm" event={"ID":"9a1a4117-0b43-4d57-8023-25bed714d499","Type":"ContainerStarted","Data":"478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675"}
Nov 03 19:49:38.062053 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:38.061890    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm" event={"ID":"9a1a4117-0b43-4d57-8023-25bed714d499","Type":"ContainerStarted","Data":"5a091c4bf8eced8dead2e1010bf4ece53d7386b52b4b551af70da3005c35f93a"}
Nov 03 19:49:38.062053 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:38.061935    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:49:38.063739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:38.063708    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr" event={"ID":"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d","Type":"ContainerStarted","Data":"b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97"}
Nov 03 19:49:38.063857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:38.063754    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr" event={"ID":"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d","Type":"ContainerStarted","Data":"d842035b88d2013f07988af9b8f8fc4aec93cb7d98325efae64b83e543381b5f"}
Nov 03 19:49:38.084696 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:38.084649    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm" podStartSLOduration=2.084636429 podStartE2EDuration="2.084636429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:38.083816532 +0000 UTC m=+1378.991838471" watchObservedRunningTime="2026-11-03 19:49:38.084636429 +0000 UTC m=+1378.992658367"
Nov 03 19:49:38.115794 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:38.115732    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr" podStartSLOduration=2.115716188 podStartE2EDuration="2.115716188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:38.110174299 +0000 UTC m=+1379.018196239" watchObservedRunningTime="2026-11-03 19:49:38.115716188 +0000 UTC m=+1379.023738128"
Nov 03 19:49:39.076345 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:39.076281    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:49:40.317437 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:40.317402    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:49:41.365681 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:41.365645    5051 scope.go:122] "RemoveContainer" containerID="5dfa8d9201e34b7f70a7a97f0e4ef240423af00735575d11aee5a3d143785bb2"
Nov 03 19:49:43.101066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:43.100285    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:49:47.569840 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.569748    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"]
Nov 03 19:49:47.577582 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.577556    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.587747 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.587704    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-300831-serving-cert\""
Nov 03 19:49:47.611838 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.594567    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"]
Nov 03 19:49:47.726074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.725010    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abcde2a0-cda0-4635-89f3-b791dee9537e-openshift-service-ca\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.726074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.725116    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-metrics-certs\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.726074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.725166    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sfkc\" (UniqueName: \"kubernetes.io/projected/abcde2a0-cda0-4635-89f3-b791dee9537e-kube-api-access-4sfkc\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.726074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.725231    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-maas-api-tls\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.826671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.826572    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abcde2a0-cda0-4635-89f3-b791dee9537e-openshift-service-ca\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.826671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.826668    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-metrics-certs\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.826904 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.826868    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sfkc\" (UniqueName: \"kubernetes.io/projected/abcde2a0-cda0-4635-89f3-b791dee9537e-kube-api-access-4sfkc\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.827006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.826973    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-maas-api-tls\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.827449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.827424    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abcde2a0-cda0-4635-89f3-b791dee9537e-openshift-service-ca\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.830307 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.829999    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-maas-api-tls\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.831545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.831521    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-metrics-certs\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.839412 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.839389    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sfkc\" (UniqueName: \"kubernetes.io/projected/abcde2a0-cda0-4635-89f3-b791dee9537e-kube-api-access-4sfkc\") pod \"maas-api-e2e-api-a-300831-7b7674495d-nx5dq\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:47.923708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:47.923297    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:48.257540 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:48.257392    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"]
Nov 03 19:49:48.257816 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:49:48.257782    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabcde2a0_cda0_4635_89f3_b791dee9537e.slice/crio-ac5b4bd9a3ed068b21c4089a76c0d1ccf8d35ae296c6f806bcded1a26e048ccf WatchSource:0}: Error finding container ac5b4bd9a3ed068b21c4089a76c0d1ccf8d35ae296c6f806bcded1a26e048ccf: Status 404 returned error can't find the container with id ac5b4bd9a3ed068b21c4089a76c0d1ccf8d35ae296c6f806bcded1a26e048ccf
Nov 03 19:49:48.319837 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:48.319792    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq" event={"ID":"abcde2a0-cda0-4635-89f3-b791dee9537e","Type":"ContainerStarted","Data":"ac5b4bd9a3ed068b21c4089a76c0d1ccf8d35ae296c6f806bcded1a26e048ccf"}
Nov 03 19:49:49.080375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.080299    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:49:49.395087 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.385093    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"]
Nov 03 19:49:49.397803 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.396724    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:49.416132 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.415273    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-8cff38-serving-cert\""
Nov 03 19:49:49.423659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.422874    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-maas-api-tls\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:49.423659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.423102    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-metrics-certs\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:49.423659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.423131    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mphgg\" (UniqueName: \"kubernetes.io/projected/526cdbab-900d-4990-9a52-deb0b1277d9a-kube-api-access-mphgg\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:49.423659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.423318    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/526cdbab-900d-4990-9a52-deb0b1277d9a-openshift-service-ca\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:49.443796 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.439255    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"]
Nov 03 19:49:49.488085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.488034    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq" event={"ID":"abcde2a0-cda0-4635-89f3-b791dee9537e","Type":"ContainerStarted","Data":"9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92"}
Nov 03 19:49:49.488327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.488306    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:49.550571 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.550451    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-maas-api-tls\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:49.552356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.550897    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-metrics-certs\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:49.552356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.550935    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mphgg\" (UniqueName: \"kubernetes.io/projected/526cdbab-900d-4990-9a52-deb0b1277d9a-kube-api-access-mphgg\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:49.552356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.551175    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/526cdbab-900d-4990-9a52-deb0b1277d9a-openshift-service-ca\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:49.552356 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:49.551695    5051 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-serving-cert: secret "maas-api-e2e-api-b-8cff38-serving-cert" not found
Nov 03 19:49:49.552356 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:49:49.551756    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-maas-api-tls podName:526cdbab-900d-4990-9a52-deb0b1277d9a nodeName:}" failed. No retries permitted until 2026-11-03 19:49:50.051734082 +0000 UTC m=+1390.959756006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-maas-api-tls") pod "maas-api-e2e-api-b-8cff38-b5545f876-bjpjv" (UID: "526cdbab-900d-4990-9a52-deb0b1277d9a") : secret "maas-api-e2e-api-b-8cff38-serving-cert" not found
Nov 03 19:49:49.554131 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.553366    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/526cdbab-900d-4990-9a52-deb0b1277d9a-openshift-service-ca\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:49.561095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.559925    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-metrics-certs\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:49.567458 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:49.567411    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mphgg\" (UniqueName: \"kubernetes.io/projected/526cdbab-900d-4990-9a52-deb0b1277d9a-kube-api-access-mphgg\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:50.077066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:50.066167    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-maas-api-tls\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:50.077066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:50.074608    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-maas-api-tls\") pod \"maas-api-e2e-api-b-8cff38-b5545f876-bjpjv\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:50.100332 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:50.100301    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:49:50.121279 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:50.121203    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq" podStartSLOduration=2.609111873 podStartE2EDuration="3.121183334s" totalImagesPullingTime="512.071461ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:49.51737378 +0000 UTC m=+1390.425395720" watchObservedRunningTime="2026-11-03 19:49:50.121183334 +0000 UTC m=+1391.029205289"
Nov 03 19:49:50.343777 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:50.342766    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:50.910120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:50.888946    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"]
Nov 03 19:49:51.510935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:51.510849    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv" event={"ID":"526cdbab-900d-4990-9a52-deb0b1277d9a","Type":"ContainerStarted","Data":"ad4993ae185a01e8cff8b3e668cc57100bc09e1dacee2dd3d205dc92fcb38cd7"}
Nov 03 19:49:51.510935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:51.510909    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv" event={"ID":"526cdbab-900d-4990-9a52-deb0b1277d9a","Type":"ContainerStarted","Data":"154c36169f28c8803f99437e333877afed3a3dc76556fbd2afdb1dcdd8e2b70b"}
Nov 03 19:49:51.511535 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:51.510952    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:51.540757 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:51.540663    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv" podStartSLOduration=2.094123992 podStartE2EDuration="2.540647223s" totalImagesPullingTime="446.523231ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:51.535310369 +0000 UTC m=+1392.443332335" watchObservedRunningTime="2026-11-03 19:49:51.540647223 +0000 UTC m=+1392.448669166"
Nov 03 19:49:52.512695 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.512648    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"]
Nov 03 19:49:52.524590 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.524558    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.525634 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.525610    5051 generic.go:356] "Generic (PLEG): container finished" podID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerID="4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2" exitCode=0
Nov 03 19:49:52.526490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.526465    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" event={"ID":"1f06217f-17b2-4745-bdf2-6ae876890f1a","Type":"ContainerDied","Data":"4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2"}
Nov 03 19:49:52.526855 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.526836    5051 scope.go:122] "RemoveContainer" containerID="4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2"
Nov 03 19:49:52.532592 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.531020    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:52.532592 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.531121    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:52.532592 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.531169    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:52.567805 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.557520    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-5444b0-openshift-default-dockercfg-4mw5s\""
Nov 03 19:49:52.567805 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.566093    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"]
Nov 03 19:49:52.690195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.686093    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-istio-token\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.690384 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.690356    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8eade3a5-b467-4599-b909-cb3306407caa-istio-podinfo\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.690448 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.690413    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-socket\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.690559 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.690537    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-data\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.690682 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.690664    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp26m\" (UniqueName: \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-kube-api-access-gp26m\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.690818 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.690802    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-credential-socket\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.690890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.690835    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-envoy\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.690890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.690868    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8eade3a5-b467-4599-b909-cb3306407caa-istiod-ca-cert\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.690995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.690906    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-certs\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.912783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.911061    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-data\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.912783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.911156    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp26m\" (UniqueName: \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-kube-api-access-gp26m\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.912783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.911271    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-credential-socket\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.912783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.911302    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-envoy\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.912783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.911337    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8eade3a5-b467-4599-b909-cb3306407caa-istiod-ca-cert\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.912783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.911372    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-certs\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.912783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.911648    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-istio-token\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.912783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.911863    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8eade3a5-b467-4599-b909-cb3306407caa-istio-podinfo\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.912783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.911905    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-socket\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.914455 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.914280    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-socket\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.914966 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.914941    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8eade3a5-b467-4599-b909-cb3306407caa-istiod-ca-cert\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.915243 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.915216    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-certs\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.917492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.916592    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-data\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.919118 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.919092    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-credential-socket\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.921349 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.921300    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-envoy\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.922869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.922847    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8eade3a5-b467-4599-b909-cb3306407caa-istio-podinfo\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.936084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.932701    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-istio-token\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:52.978646 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:52.962493    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp26m\" (UniqueName: \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-kube-api-access-gp26m\") pod \"e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") " pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:53.186724 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.186692    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:53.502236 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:49:53.502194    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eade3a5_b467_4599_b909_cb3306407caa.slice/crio-793d1ccdbe145788a9a4cee72066d4d9bc76383ec7ef0879788bdb97987bb0fb WatchSource:0}: Error finding container 793d1ccdbe145788a9a4cee72066d4d9bc76383ec7ef0879788bdb97987bb0fb: Status 404 returned error can't find the container with id 793d1ccdbe145788a9a4cee72066d4d9bc76383ec7ef0879788bdb97987bb0fb
Nov 03 19:49:53.505311 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.505287    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"]
Nov 03 19:49:53.505853 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.505824    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:53.505899 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.505889    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:53.505934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.505919    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:53.545971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.545940    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" event={"ID":"1f06217f-17b2-4745-bdf2-6ae876890f1a","Type":"ContainerStarted","Data":"19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782"}
Nov 03 19:49:53.548368 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.548318    5051 generic.go:356] "Generic (PLEG): container finished" podID="262d3eee-5594-4d75-83e2-1eb72166f710" containerID="eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6" exitCode=0
Nov 03 19:49:53.548488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.548433    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" event={"ID":"262d3eee-5594-4d75-83e2-1eb72166f710","Type":"ContainerDied","Data":"eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6"}
Nov 03 19:49:53.548754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.548738    5051 scope.go:122] "RemoveContainer" containerID="eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6"
Nov 03 19:49:53.551523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.550706    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:53.551523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.550784    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:53.551523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.550825    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:49:53.552486 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:53.552208    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" event={"ID":"8eade3a5-b467-4599-b909-cb3306407caa","Type":"ContainerStarted","Data":"793d1ccdbe145788a9a4cee72066d4d9bc76383ec7ef0879788bdb97987bb0fb"}
Nov 03 19:49:54.235633 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:54.235604    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:49:54.236698 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:54.236673    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 03 19:49:54.236774 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:54.236724    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 03 19:49:54.577790 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:54.577691    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" event={"ID":"262d3eee-5594-4d75-83e2-1eb72166f710","Type":"ContainerStarted","Data":"80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543"}
Nov 03 19:49:54.583216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:54.583168    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" event={"ID":"8eade3a5-b467-4599-b909-cb3306407caa","Type":"ContainerStarted","Data":"7337569bc2f97c82612abf258988701a6ad700826c67d1c0007ebe7bfecb4bd8"}
Nov 03 19:49:54.636185 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:54.636128    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" podStartSLOduration=2.636112651 podStartE2EDuration="2.636112651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:49:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:49:54.633506705 +0000 UTC m=+1395.541528644" watchObservedRunningTime="2026-11-03 19:49:54.636112651 +0000 UTC m=+1395.544134590"
Nov 03 19:49:55.187066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:55.176559    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:49:55.187066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:55.178474    5051 patch_prober.go:29] interesting pod/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 03 19:49:55.187066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:55.178525    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 03 19:49:55.209064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:55.198862    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:55.236129 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:55.236098    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 03 19:49:55.236315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:55.236145    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 03 19:49:55.239428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:55.239404    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:55.504001 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:55.503966    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:49:55.609346 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:55.609314    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:55.610304 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:55.610283    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:49:56.164066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.163442    5051 patch_prober.go:29] interesting pod/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 03 19:49:56.164066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.163510    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 03 19:49:56.211197 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.211167    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"]
Nov 03 19:49:56.211596 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.211569    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy" containerID="cri-o://5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce" gracePeriod=30
Nov 03 19:49:56.216432 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.216311    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"]
Nov 03 19:49:56.216573 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.216516    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" containerID="cri-o://0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4" gracePeriod=30
Nov 03 19:49:56.220607 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.220583    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"]
Nov 03 19:49:56.220777 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.220755    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" containerID="cri-o://0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8" gracePeriod=30
Nov 03 19:49:56.235613 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.235579    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 03 19:49:56.235741 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.235631    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 03 19:49:56.309052 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.308998    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"]
Nov 03 19:49:56.309721 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.309618    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy" containerID="cri-o://42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30" gracePeriod=30
Nov 03 19:49:56.316362 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.316332    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"]
Nov 03 19:49:56.316854 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.316571    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy" containerID="cri-o://2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f" gracePeriod=30
Nov 03 19:49:56.321513 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.321486    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"]
Nov 03 19:49:56.321658 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:56.321641    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy" containerID="cri-o://6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f" gracePeriod=30
Nov 03 19:49:57.164065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:57.162487    5051 patch_prober.go:29] interesting pod/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 03 19:49:57.164065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:57.162560    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 03 19:49:57.236301 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:57.236241    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 03 19:49:57.236301 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:57.236293    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 03 19:49:57.531982 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:57.531949    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:49:57.714381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:57.714331    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 19:49:57.714575 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:57.714405    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 19:49:57.715748 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:57.715715    5051 patch_prober.go:29] interesting pod/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 19:49:57.715896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:57.715772    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 19:49:57.838694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:57.838623    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:49:58.163591 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:58.162401    5051 patch_prober.go:29] interesting pod/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 03 19:49:58.163591 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:58.162477    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 03 19:49:58.235521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:58.235492    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 03 19:49:58.235942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:58.235539    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 03 19:49:59.147184 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:59.147146    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:49:59.167077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:59.165493    5051 patch_prober.go:29] interesting pod/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 03 19:49:59.167077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:59.165552    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 03 19:49:59.238502 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:59.238475    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 03 19:49:59.239064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:49:59.239015    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 03 19:50:00.163962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:00.163917    5051 patch_prober.go:29] interesting pod/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 03 19:50:00.164181 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:00.163978    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 03 19:50:00.187855 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:00.187825    5051 patch_prober.go:29] interesting pod/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 19:50:00.188002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:00.187886    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 19:50:00.256065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:00.251404    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:50:00.256065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:00.255629    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:50:01.166266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.166231    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:50:01.168412 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.168385    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:50:01.607030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.607004    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:50:01.610591 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.610558    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677473    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13925e4e-1b1f-4306-af1d-418c993e3fb8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13925e4e-1b1f-4306-af1d-418c993e3fb8-istiod-ca-cert\") pod \"13925e4e-1b1f-4306-af1d-418c993e3fb8\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677537    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-credential-socket\") pod \"13925e4e-1b1f-4306-af1d-418c993e3fb8\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677581    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-data\") pod \"d276c115-0214-490a-bcd7-206b3beed9d4\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677636    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-envoy\") pod \"d276c115-0214-490a-bcd7-206b3beed9d4\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677684    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-socket\") pod \"d276c115-0214-490a-bcd7-206b3beed9d4\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677716    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-kube-api-access-hb7lj\" (UniqueName: \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-kube-api-access-hb7lj\") pod \"d276c115-0214-490a-bcd7-206b3beed9d4\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677757    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-certs\") pod \"d276c115-0214-490a-bcd7-206b3beed9d4\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677807    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-token\" (UniqueName: \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-token\") pod \"13925e4e-1b1f-4306-af1d-418c993e3fb8\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677920    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-kube-api-access-bp7hn\" (UniqueName: \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-kube-api-access-bp7hn\") pod \"13925e4e-1b1f-4306-af1d-418c993e3fb8\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677948    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-credential-socket\") pod \"d276c115-0214-490a-bcd7-206b3beed9d4\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677958    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-data" pod "d276c115-0214-490a-bcd7-206b3beed9d4" (UID: "d276c115-0214-490a-bcd7-206b3beed9d4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677976    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-istio-token\" (UniqueName: \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-istio-token\") pod \"d276c115-0214-490a-bcd7-206b3beed9d4\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.677902    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13925e4e-1b1f-4306-af1d-418c993e3fb8-istiod-ca-cert" pod "13925e4e-1b1f-4306-af1d-418c993e3fb8" (UID: "13925e4e-1b1f-4306-af1d-418c993e3fb8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.678013    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-podinfo\") pod \"13925e4e-1b1f-4306-af1d-418c993e3fb8\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.678060    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-socket\") pod \"13925e4e-1b1f-4306-af1d-418c993e3fb8\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") "
Nov 03 19:50:01.678093 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.678102    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-data\") pod \"13925e4e-1b1f-4306-af1d-418c993e3fb8\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") "
Nov 03 19:50:01.679067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.678119    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-envoy\") pod \"13925e4e-1b1f-4306-af1d-418c993e3fb8\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") "
Nov 03 19:50:01.679067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.678137    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d276c115-0214-490a-bcd7-206b3beed9d4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d276c115-0214-490a-bcd7-206b3beed9d4-istio-podinfo\") pod \"d276c115-0214-490a-bcd7-206b3beed9d4\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") "
Nov 03 19:50:01.679067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.678157    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d276c115-0214-490a-bcd7-206b3beed9d4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d276c115-0214-490a-bcd7-206b3beed9d4-istiod-ca-cert\") pod \"d276c115-0214-490a-bcd7-206b3beed9d4\" (UID: \"d276c115-0214-490a-bcd7-206b3beed9d4\") "
Nov 03 19:50:01.679067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.678170    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-certs" pod "d276c115-0214-490a-bcd7-206b3beed9d4" (UID: "d276c115-0214-490a-bcd7-206b3beed9d4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.679067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.678328    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-socket" pod "d276c115-0214-490a-bcd7-206b3beed9d4" (UID: "d276c115-0214-490a-bcd7-206b3beed9d4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.679067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.678447    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-certs\") pod \"13925e4e-1b1f-4306-af1d-418c993e3fb8\" (UID: \"13925e4e-1b1f-4306-af1d-418c993e3fb8\") "
Nov 03 19:50:01.679067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.678549    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d276c115-0214-490a-bcd7-206b3beed9d4-istiod-ca-cert" pod "d276c115-0214-490a-bcd7-206b3beed9d4" (UID: "d276c115-0214-490a-bcd7-206b3beed9d4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:01.679067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.678722    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-credential-socket" pod "d276c115-0214-490a-bcd7-206b3beed9d4" (UID: "d276c115-0214-490a-bcd7-206b3beed9d4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.679550 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.679221    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13925e4e-1b1f-4306-af1d-418c993e3fb8-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.679550 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.679241    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.679550 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.679257    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.679550 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.679270    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.679550 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.679282    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.679550 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.679295    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d276c115-0214-490a-bcd7-206b3beed9d4-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.679859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.679666    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-credential-socket" pod "13925e4e-1b1f-4306-af1d-418c993e3fb8" (UID: "13925e4e-1b1f-4306-af1d-418c993e3fb8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.680835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.680181    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-certs" pod "13925e4e-1b1f-4306-af1d-418c993e3fb8" (UID: "13925e4e-1b1f-4306-af1d-418c993e3fb8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.680835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.680194    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-data" pod "13925e4e-1b1f-4306-af1d-418c993e3fb8" (UID: "13925e4e-1b1f-4306-af1d-418c993e3fb8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.680835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.680467    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-socket" pod "13925e4e-1b1f-4306-af1d-418c993e3fb8" (UID: "13925e4e-1b1f-4306-af1d-418c993e3fb8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.684267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.683602    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-envoy" pod "13925e4e-1b1f-4306-af1d-418c993e3fb8" (UID: "13925e4e-1b1f-4306-af1d-418c993e3fb8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.684267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.683697    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-kube-api-access-hb7lj" pod "d276c115-0214-490a-bcd7-206b3beed9d4" (UID: "d276c115-0214-490a-bcd7-206b3beed9d4"). InnerVolumeSpecName "kube-api-access-hb7lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.684427 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.684344    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-token" pod "13925e4e-1b1f-4306-af1d-418c993e3fb8" (UID: "13925e4e-1b1f-4306-af1d-418c993e3fb8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.685882 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.685859    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-podinfo" pod "13925e4e-1b1f-4306-af1d-418c993e3fb8" (UID: "13925e4e-1b1f-4306-af1d-418c993e3fb8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:50:01.686842 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.686819    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d276c115-0214-490a-bcd7-206b3beed9d4-istio-podinfo" pod "d276c115-0214-490a-bcd7-206b3beed9d4" (UID: "d276c115-0214-490a-bcd7-206b3beed9d4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:50:01.687400 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.687375    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-envoy" pod "d276c115-0214-490a-bcd7-206b3beed9d4" (UID: "d276c115-0214-490a-bcd7-206b3beed9d4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.687508 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.687429    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-kube-api-access-bp7hn" pod "13925e4e-1b1f-4306-af1d-418c993e3fb8" (UID: "13925e4e-1b1f-4306-af1d-418c993e3fb8"). InnerVolumeSpecName "kube-api-access-bp7hn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.689083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.689022    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-istio-token" pod "d276c115-0214-490a-bcd7-206b3beed9d4" (UID: "d276c115-0214-490a-bcd7-206b3beed9d4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.704683 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.704462    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:50:01.780075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.780020    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istiod-ca-cert\") pod \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") "
Nov 03 19:50:01.780322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.780127    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-token\" (UniqueName: \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-token\") pod \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") "
Nov 03 19:50:01.780322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.780196    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-certs\") pod \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") "
Nov 03 19:50:01.780322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.780248    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-data\") pod \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") "
Nov 03 19:50:01.780568 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.780407    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-credential-socket\") pod \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") "
Nov 03 19:50:01.780568 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.780475    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-podinfo\") pod \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") "
Nov 03 19:50:01.780568 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.780536    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-envoy\") pod \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") "
Nov 03 19:50:01.780779 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.780597    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istiod-ca-cert" pod "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" (UID: "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:01.780779 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.780632    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-socket\") pod \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") "
Nov 03 19:50:01.780926 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.780811    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-kube-api-access-8hpf7\" (UniqueName: \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-kube-api-access-8hpf7\") pod \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\" (UID: \"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f\") "
Nov 03 19:50:01.780995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.780966    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-data" pod "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" (UID: "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.781677 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781628    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-certs" pod "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" (UID: "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.781677 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781658    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp7hn\" (UniqueName: \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-kube-api-access-bp7hn\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781684    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781698    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781711    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781725    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781737    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781749    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d276c115-0214-490a-bcd7-206b3beed9d4-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781763    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781776    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781793    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13925e4e-1b1f-4306-af1d-418c993e3fb8-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781807    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d276c115-0214-490a-bcd7-206b3beed9d4-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781821    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb7lj\" (UniqueName: \"kubernetes.io/projected/d276c115-0214-490a-bcd7-206b3beed9d4-kube-api-access-hb7lj\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781834    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13925e4e-1b1f-4306-af1d-418c993e3fb8-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.781857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.781848    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.782535 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.782272    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-credential-socket" pod "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" (UID: "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.782535 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.782388    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-socket" pod "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" (UID: "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.784956 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.784929    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-token" pod "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" (UID: "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.786488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.786464    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-envoy" pod "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" (UID: "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.787964 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.787937    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-kube-api-access-8hpf7" pod "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" (UID: "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f"). InnerVolumeSpecName "kube-api-access-8hpf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.789446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.789419    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-podinfo" pod "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" (UID: "1ee9dca8-9896-4cda-b5e4-6c63aa7a268f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:50:01.801369 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.801342    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:50:01.805316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.805296    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:50:01.810718 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.810700    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:50:01.885669 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.885637    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-socket\") pod \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") "
Nov 03 19:50:01.885857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.885701    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istiod-ca-cert\") pod \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") "
Nov 03 19:50:01.885857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.885750    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-podinfo\") pod \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") "
Nov 03 19:50:01.885857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.885811    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-kube-api-access-h25z8\" (UniqueName: \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-kube-api-access-h25z8\") pod \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") "
Nov 03 19:50:01.885857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.885846    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-credential-socket\") pod \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") "
Nov 03 19:50:01.886135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.885879    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-podinfo\") pod \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") "
Nov 03 19:50:01.886135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.885915    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-certs\") pod \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") "
Nov 03 19:50:01.886135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.885962    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-certs\") pod \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") "
Nov 03 19:50:01.886135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.885981    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-socket" pod "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" (UID: "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.886135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.885995    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-token\" (UniqueName: \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-token\") pod \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") "
Nov 03 19:50:01.886135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886054    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-envoy\") pod \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") "
Nov 03 19:50:01.886135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886094    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-credential-socket\") pod \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") "
Nov 03 19:50:01.886135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886122    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-data\") pod \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") "
Nov 03 19:50:01.886539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886183    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-envoy\") pod \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") "
Nov 03 19:50:01.886539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886254    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-socket\") pod \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") "
Nov 03 19:50:01.886539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886283    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-credential-socket\") pod \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") "
Nov 03 19:50:01.886539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886343    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-podinfo\") pod \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") "
Nov 03 19:50:01.886539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886357    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-credential-socket" pod "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" (UID: "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.886539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886384    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istiod-ca-cert" pod "c2d6acc6-3543-4cd5-ac03-38a9b412f25b" (UID: "c2d6acc6-3543-4cd5-ac03-38a9b412f25b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:01.886539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886485    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-kube-api-access-dl4kx\" (UniqueName: \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-kube-api-access-dl4kx\") pod \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") "
Nov 03 19:50:01.887382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886552    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-token\" (UniqueName: \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-token\") pod \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") "
Nov 03 19:50:01.887382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886602    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-kube-api-access-wlmsg\" (UniqueName: \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-kube-api-access-wlmsg\") pod \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") "
Nov 03 19:50:01.887382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886648    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-data\") pod \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") "
Nov 03 19:50:01.887382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886733    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-data\") pod \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") "
Nov 03 19:50:01.887382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.886768    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-token\" (UniqueName: \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-token\") pod \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") "
Nov 03 19:50:01.887382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.887198    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-socket\") pod \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") "
Nov 03 19:50:01.887382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.887241    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-certs\") pod \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\" (UID: \"c2d6acc6-3543-4cd5-ac03-38a9b412f25b\") "
Nov 03 19:50:01.887382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.887272    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-envoy\") pod \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") "
Nov 03 19:50:01.887382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.887348    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istiod-ca-cert\") pod \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\" (UID: \"bd2e2038-3d81-4246-8ef8-c29bfb7e8942\") "
Nov 03 19:50:01.887382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.887383    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istiod-ca-cert\") pod \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\" (UID: \"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa\") "
Nov 03 19:50:01.887851 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.887468    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-data" pod "c2d6acc6-3543-4cd5-ac03-38a9b412f25b" (UID: "c2d6acc6-3543-4cd5-ac03-38a9b412f25b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.887851 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.887695    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-certs" pod "bd2e2038-3d81-4246-8ef8-c29bfb7e8942" (UID: "bd2e2038-3d81-4246-8ef8-c29bfb7e8942"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.888769 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.888666    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-data" pod "bd2e2038-3d81-4246-8ef8-c29bfb7e8942" (UID: "bd2e2038-3d81-4246-8ef8-c29bfb7e8942"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.888916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.888890    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.888920    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.888935    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.888949    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hpf7\" (UniqueName: \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-kube-api-access-8hpf7\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.888964    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.888978    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.888992    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.889006    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.889020    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.889034    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.889061    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.889074    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.889386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.889087    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.891218 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.891193    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-token" pod "c2d6acc6-3543-4cd5-ac03-38a9b412f25b" (UID: "c2d6acc6-3543-4cd5-ac03-38a9b412f25b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.891542 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.891517    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istiod-ca-cert" pod "bd2e2038-3d81-4246-8ef8-c29bfb7e8942" (UID: "bd2e2038-3d81-4246-8ef8-c29bfb7e8942"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:01.891877 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.891852    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istiod-ca-cert" pod "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" (UID: "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:01.891956 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.891918    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-podinfo" pod "c2d6acc6-3543-4cd5-ac03-38a9b412f25b" (UID: "c2d6acc6-3543-4cd5-ac03-38a9b412f25b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:50:01.892638 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.892392    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-certs" pod "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" (UID: "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.894103 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.893408    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-envoy" pod "c2d6acc6-3543-4cd5-ac03-38a9b412f25b" (UID: "c2d6acc6-3543-4cd5-ac03-38a9b412f25b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.894103 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.893687    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-socket" pod "bd2e2038-3d81-4246-8ef8-c29bfb7e8942" (UID: "bd2e2038-3d81-4246-8ef8-c29bfb7e8942"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.894103 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.893909    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-credential-socket" pod "c2d6acc6-3543-4cd5-ac03-38a9b412f25b" (UID: "c2d6acc6-3543-4cd5-ac03-38a9b412f25b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.894310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.894190    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-socket" pod "c2d6acc6-3543-4cd5-ac03-38a9b412f25b" (UID: "c2d6acc6-3543-4cd5-ac03-38a9b412f25b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.894310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.894231    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-podinfo" pod "bd2e2038-3d81-4246-8ef8-c29bfb7e8942" (UID: "bd2e2038-3d81-4246-8ef8-c29bfb7e8942"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:50:01.894472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.894452    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-credential-socket" pod "bd2e2038-3d81-4246-8ef8-c29bfb7e8942" (UID: "bd2e2038-3d81-4246-8ef8-c29bfb7e8942"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.894592 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.894549    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-token" pod "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" (UID: "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.894699 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.894620    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-data" pod "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" (UID: "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.894699 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.894619    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-kube-api-access-wlmsg" pod "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" (UID: "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa"). InnerVolumeSpecName "kube-api-access-wlmsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.894699 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.894666    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-envoy" pod "bd2e2038-3d81-4246-8ef8-c29bfb7e8942" (UID: "bd2e2038-3d81-4246-8ef8-c29bfb7e8942"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.894868 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.894825    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-certs" pod "c2d6acc6-3543-4cd5-ac03-38a9b412f25b" (UID: "c2d6acc6-3543-4cd5-ac03-38a9b412f25b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.895320 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.895296    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-kube-api-access-h25z8" pod "c2d6acc6-3543-4cd5-ac03-38a9b412f25b" (UID: "c2d6acc6-3543-4cd5-ac03-38a9b412f25b"). InnerVolumeSpecName "kube-api-access-h25z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.897351 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.897312    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-podinfo" pod "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" (UID: "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:50:01.897439 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.897420    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-kube-api-access-dl4kx" pod "bd2e2038-3d81-4246-8ef8-c29bfb7e8942" (UID: "bd2e2038-3d81-4246-8ef8-c29bfb7e8942"). InnerVolumeSpecName "kube-api-access-dl4kx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.897953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.897931    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-envoy" pod "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" (UID: "4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:01.898238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.898213    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-token" pod "bd2e2038-3d81-4246-8ef8-c29bfb7e8942" (UID: "bd2e2038-3d81-4246-8ef8-c29bfb7e8942"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:01.972897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.972872    5051 generic.go:356] "Generic (PLEG): container finished" podID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerID="2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f" exitCode=0
Nov 03 19:50:01.973022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.972922    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"
Nov 03 19:50:01.973022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.972952    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" event={"ID":"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa","Type":"ContainerDied","Data":"2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f"}
Nov 03 19:50:01.973137 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.973073    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm" event={"ID":"4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa","Type":"ContainerDied","Data":"b7bbcc5d81a896e64c141272dfb294e778bf2f6615a783d20e3173b9a4049260"}
Nov 03 19:50:01.973137 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.973104    5051 scope.go:122] "RemoveContainer" containerID="2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f"
Nov 03 19:50:01.974539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.974505    5051 generic.go:356] "Generic (PLEG): container finished" podID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerID="0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8" exitCode=0
Nov 03 19:50:01.974629 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.974550    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"
Nov 03 19:50:01.974629 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.974562    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" event={"ID":"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f","Type":"ContainerDied","Data":"0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8"}
Nov 03 19:50:01.974629 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.974611    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7" event={"ID":"1ee9dca8-9896-4cda-b5e4-6c63aa7a268f","Type":"ContainerDied","Data":"4edced7c46c580e292927878ba327485ed34a6879939ac865f03a1cddda7de69"}
Nov 03 19:50:01.976311 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.976289    5051 generic.go:356] "Generic (PLEG): container finished" podID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerID="42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30" exitCode=0
Nov 03 19:50:01.976428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.976350    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"
Nov 03 19:50:01.976428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.976385    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" event={"ID":"c2d6acc6-3543-4cd5-ac03-38a9b412f25b","Type":"ContainerDied","Data":"42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30"}
Nov 03 19:50:01.976428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.976420    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz" event={"ID":"c2d6acc6-3543-4cd5-ac03-38a9b412f25b","Type":"ContainerDied","Data":"d1f0ca1b09bcc4b709dc8e2f7458c13dc2653eea28e816f7c55fbc7a9a5d08bc"}
Nov 03 19:50:01.978136 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.978111    5051 generic.go:356] "Generic (PLEG): container finished" podID="d276c115-0214-490a-bcd7-206b3beed9d4" containerID="5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce" exitCode=0
Nov 03 19:50:01.978248 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.978161    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"
Nov 03 19:50:01.978248 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.978178    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" event={"ID":"d276c115-0214-490a-bcd7-206b3beed9d4","Type":"ContainerDied","Data":"5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce"}
Nov 03 19:50:01.978248 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.978225    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2" event={"ID":"d276c115-0214-490a-bcd7-206b3beed9d4","Type":"ContainerDied","Data":"5c5d21749c77e960713e747ec6ea1c93b9eb32db6e8fabb518688db8b098a709"}
Nov 03 19:50:01.985290 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.985268    5051 generic.go:356] "Generic (PLEG): container finished" podID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerID="0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4" exitCode=0
Nov 03 19:50:01.985393 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.985344    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" event={"ID":"13925e4e-1b1f-4306-af1d-418c993e3fb8","Type":"ContainerDied","Data":"0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4"}
Nov 03 19:50:01.985607 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.985391    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw" event={"ID":"13925e4e-1b1f-4306-af1d-418c993e3fb8","Type":"ContainerDied","Data":"3e94e0bc7286ec430265463b7d69b38da5649f7aec88581bcff2cc76381b5293"}
Nov 03 19:50:01.985924 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.985820    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"
Nov 03 19:50:01.987468 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.987430    5051 scope.go:122] "RemoveContainer" containerID="2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f"
Nov 03 19:50:01.987769 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.987750    5051 generic.go:356] "Generic (PLEG): container finished" podID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerID="6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f" exitCode=0
Nov 03 19:50:01.987877 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:01.987852    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f\": container with ID starting with 2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f not found: ID does not exist" containerID="2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f"
Nov 03 19:50:01.987942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.987878    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" event={"ID":"bd2e2038-3d81-4246-8ef8-c29bfb7e8942","Type":"ContainerDied","Data":"6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f"}
Nov 03 19:50:01.987942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.987883    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f"} err="failed to get container status \"2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f\": rpc error: code = NotFound desc = could not find container \"2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f\": container with ID starting with 2c2e52747b74c672c0db7c96dd36bef4817954209953734ebc2fcda928e6a31f not found: ID does not exist"
Nov 03 19:50:01.987942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.987906    5051 scope.go:122] "RemoveContainer" containerID="0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8"
Nov 03 19:50:01.987942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.987915    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"
Nov 03 19:50:01.987942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.987922    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7" event={"ID":"bd2e2038-3d81-4246-8ef8-c29bfb7e8942","Type":"ContainerDied","Data":"f1476fca16202e4027b60b1bcaf855e713a0cc9ab4ce4c4d2c3a398c6f1b1bf5"}
Nov 03 19:50:01.990421 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990402    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990425    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h25z8\" (UniqueName: \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-kube-api-access-h25z8\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990443    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990458    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990471    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990485    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990499    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990512    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990525    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990539    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990553    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990566    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990587    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl4kx\" (UniqueName: \"kubernetes.io/projected/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-kube-api-access-dl4kx\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990601    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990615    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlmsg\" (UniqueName: \"kubernetes.io/projected/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-kube-api-access-wlmsg\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990629    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990643    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990657    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2d6acc6-3543-4cd5-ac03-38a9b412f25b-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990670    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990684    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd2e2038-3d81-4246-8ef8-c29bfb7e8942-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:01.990762 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:01.990717    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:02.016145 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.015885    5051 scope.go:122] "RemoveContainer" containerID="0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8"
Nov 03 19:50:02.016446 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:02.016414    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8\": container with ID starting with 0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8 not found: ID does not exist" containerID="0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8"
Nov 03 19:50:02.016580 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.016452    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8"} err="failed to get container status \"0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8\": rpc error: code = NotFound desc = could not find container \"0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8\": container with ID starting with 0a269752cf5e7ed5c6f1680aa8ca1ed27919b6c9f2d7ba722c59c01eed4169d8 not found: ID does not exist"
Nov 03 19:50:02.016580 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.016475    5051 scope.go:122] "RemoveContainer" containerID="42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30"
Nov 03 19:50:02.036650 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.036620    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"]
Nov 03 19:50:02.036802 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.036774    5051 scope.go:122] "RemoveContainer" containerID="42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30"
Nov 03 19:50:02.037236 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:02.037206    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30\": container with ID starting with 42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30 not found: ID does not exist" containerID="42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30"
Nov 03 19:50:02.037350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.037247    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30"} err="failed to get container status \"42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30\": rpc error: code = NotFound desc = could not find container \"42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30\": container with ID starting with 42834ac9d2683bd0cf3e3cb7a52ae6940dbde4ed5bb0e8ff8147776f79d5ee30 not found: ID does not exist"
Nov 03 19:50:02.037350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.037270    5051 scope.go:122] "RemoveContainer" containerID="5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce"
Nov 03 19:50:02.041802 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.041777    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-vt2dm"]
Nov 03 19:50:02.051548 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.051523    5051 scope.go:122] "RemoveContainer" containerID="5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce"
Nov 03 19:50:02.051884 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:02.051852    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce\": container with ID starting with 5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce not found: ID does not exist" containerID="5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce"
Nov 03 19:50:02.051998 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.051893    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce"} err="failed to get container status \"5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce\": rpc error: code = NotFound desc = could not find container \"5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce\": container with ID starting with 5be657f1b7e9d0284fc1b76525c618d64669aa4a871bc6f3145e867c8e7960ce not found: ID does not exist"
Nov 03 19:50:02.051998 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.051934    5051 scope.go:122] "RemoveContainer" containerID="0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4"
Nov 03 19:50:02.074631 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.073850    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"]
Nov 03 19:50:02.075971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.075943    5051 scope.go:122] "RemoveContainer" containerID="0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4"
Nov 03 19:50:02.076435 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:02.076410    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4\": container with ID starting with 0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4 not found: ID does not exist" containerID="0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4"
Nov 03 19:50:02.076511 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.076447    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4"} err="failed to get container status \"0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4\": rpc error: code = NotFound desc = could not find container \"0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4\": container with ID starting with 0924ee1c869bd15d524f8852409496235a03b2d0b07f1504bcec591d96c920e4 not found: ID does not exist"
Nov 03 19:50:02.076511 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.076471    5051 scope.go:122] "RemoveContainer" containerID="6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f"
Nov 03 19:50:02.079094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.079026    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-7qck7"]
Nov 03 19:50:02.090398 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.090371    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"]
Nov 03 19:50:02.093132 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.093113    5051 scope.go:122] "RemoveContainer" containerID="ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237"
Nov 03 19:50:02.103015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.102981    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-pdzv2"]
Nov 03 19:50:02.103513 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.103490    5051 scope.go:122] "RemoveContainer" containerID="6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f"
Nov 03 19:50:02.103823 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:02.103803    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f\": container with ID starting with 6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f not found: ID does not exist" containerID="6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f"
Nov 03 19:50:02.103898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.103828    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f"} err="failed to get container status \"6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f\": rpc error: code = NotFound desc = could not find container \"6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f\": container with ID starting with 6ba866ec392d09ca904f295af1a19df4c49c20f27c6d929619a3ade5e960e06f not found: ID does not exist"
Nov 03 19:50:02.103898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.103845    5051 scope.go:122] "RemoveContainer" containerID="ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237"
Nov 03 19:50:02.104141 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:02.104122    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237\": container with ID starting with ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237 not found: ID does not exist" containerID="ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237"
Nov 03 19:50:02.104203 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.104146    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237"} err="failed to get container status \"ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237\": rpc error: code = NotFound desc = could not find container \"ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237\": container with ID starting with ba97e71a37746de33efe173e6ee1e822f73f283918991eb47eb4379543cae237 not found: ID does not exist"
Nov 03 19:50:02.117943 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.117916    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"]
Nov 03 19:50:02.121612 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.121593    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-2swqw"]
Nov 03 19:50:02.133737 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.133717    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"]
Nov 03 19:50:02.136960 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.136942    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-ncscz"]
Nov 03 19:50:02.148304 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.148281    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"]
Nov 03 19:50:02.153504 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:02.153479    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-h6cs7"]
Nov 03 19:50:02.783555 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:02.783521    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:50:03.230681 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:03.230649    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" path="/var/lib/kubelet/pods/13925e4e-1b1f-4306-af1d-418c993e3fb8/volumes"
Nov 03 19:50:03.231197 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:03.231182    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" path="/var/lib/kubelet/pods/1ee9dca8-9896-4cda-b5e4-6c63aa7a268f/volumes"
Nov 03 19:50:03.231757 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:03.231734    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" path="/var/lib/kubelet/pods/4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa/volumes"
Nov 03 19:50:03.232489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:03.232467    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" path="/var/lib/kubelet/pods/bd2e2038-3d81-4246-8ef8-c29bfb7e8942/volumes"
Nov 03 19:50:03.232946 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:03.232928    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" path="/var/lib/kubelet/pods/c2d6acc6-3543-4cd5-ac03-38a9b412f25b/volumes"
Nov 03 19:50:03.233431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:03.233416    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" path="/var/lib/kubelet/pods/d276c115-0214-490a-bcd7-206b3beed9d4/volumes"
Nov 03 19:50:05.534218 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.534178    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"]
Nov 03 19:50:05.535374 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535312    5051 cpu_manager.go:414] "containerMap: removing container" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy"
Nov 03 19:50:05.535556 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535530    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy"
Nov 03 19:50:05.535647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535561    5051 cpu_manager.go:414] "containerMap: removing container" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy"
Nov 03 19:50:05.535647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535570    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy"
Nov 03 19:50:05.535647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535582    5051 cpu_manager.go:414] "containerMap: removing container" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy"
Nov 03 19:50:05.535647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535590    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy"
Nov 03 19:50:05.535647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535614    5051 cpu_manager.go:414] "containerMap: removing container" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy"
Nov 03 19:50:05.535647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535621    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy"
Nov 03 19:50:05.535647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535634    5051 cpu_manager.go:414] "containerMap: removing container" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy"
Nov 03 19:50:05.535647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535641    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy"
Nov 03 19:50:05.536030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535693    5051 cpu_manager.go:414] "containerMap: removing container" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy"
Nov 03 19:50:05.536030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535701    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy"
Nov 03 19:50:05.536030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535948    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ee9dca8-9896-4cda-b5e4-6c63aa7a268f" containerName="istio-proxy"
Nov 03 19:50:05.536030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535967    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy"
Nov 03 19:50:05.536030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535983    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy"
Nov 03 19:50:05.536030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.535993    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2d6acc6-3543-4cd5-ac03-38a9b412f25b" containerName="istio-proxy"
Nov 03 19:50:05.536030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.536003    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="13925e4e-1b1f-4306-af1d-418c993e3fb8" containerName="istio-proxy"
Nov 03 19:50:05.536030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.536013    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="4aea9ee4-a9ae-48f5-aeb8-b6a893b70caa" containerName="istio-proxy"
Nov 03 19:50:05.536030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.536024    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="d276c115-0214-490a-bcd7-206b3beed9d4" containerName="istio-proxy"
Nov 03 19:50:05.542701 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.542679    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.551991 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.551967    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"]
Nov 03 19:50:05.562109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.562088    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-9ce34d-openshift-default-dockercfg-xxssd\""
Nov 03 19:50:05.646446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.646414    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-envoy\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.646643 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.646456    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7655d6f-386a-4364-9343-6e56877d3faa-istio-podinfo\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.646742 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.646719    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-istio-token\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.646863 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.646842    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdgzl\" (UniqueName: \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-kube-api-access-fdgzl\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.646931 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.646918    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-data\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.647011 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.646992    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-socket\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.647134 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.647119    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7655d6f-386a-4364-9343-6e56877d3faa-istiod-ca-cert\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.647222 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.647208    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-certs\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.647296 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.647282    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-credential-socket\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.749123 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.749088    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-istio-token\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.749325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.749182    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdgzl\" (UniqueName: \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-kube-api-access-fdgzl\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.749325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.749267    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-data\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.749325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.749297    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-socket\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.750036 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.749361    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7655d6f-386a-4364-9343-6e56877d3faa-istiod-ca-cert\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.750036 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.749429    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-certs\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.750036 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.749488    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-credential-socket\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.750036 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.749527    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-envoy\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.750036 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.749562    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7655d6f-386a-4364-9343-6e56877d3faa-istio-podinfo\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.750036 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.749788    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-data\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.750643 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.750095    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-certs\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.750643 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.750121    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-socket\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.750643 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.750299    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-credential-socket\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.750760 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.750731    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7655d6f-386a-4364-9343-6e56877d3faa-istiod-ca-cert\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.752578 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.752553    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7655d6f-386a-4364-9343-6e56877d3faa-istio-podinfo\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.752770 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.752751    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-envoy\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.761661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.761638    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-istio-token\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.761916 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.761899    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdgzl\" (UniqueName: \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-kube-api-access-fdgzl\") pod \"e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") " pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:05.859511 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:05.859426    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:06.168916 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:50:06.168886    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7655d6f_386a_4364_9343_6e56877d3faa.slice/crio-b7c48c89210cdc178e05dfaaacd6354211504a214c59ecdb95a2013e871c53e8 WatchSource:0}: Error finding container b7c48c89210cdc178e05dfaaacd6354211504a214c59ecdb95a2013e871c53e8: Status 404 returned error can't find the container with id b7c48c89210cdc178e05dfaaacd6354211504a214c59ecdb95a2013e871c53e8
Nov 03 19:50:06.169062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:06.168982    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"]
Nov 03 19:50:06.172621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:06.172580    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:06.172726 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:06.172664    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:06.172726 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:06.172710    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:07.042842 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:07.042802    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" event={"ID":"d7655d6f-386a-4364-9343-6e56877d3faa","Type":"ContainerStarted","Data":"219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2"}
Nov 03 19:50:07.043244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:07.042873    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" event={"ID":"d7655d6f-386a-4364-9343-6e56877d3faa","Type":"ContainerStarted","Data":"b7c48c89210cdc178e05dfaaacd6354211504a214c59ecdb95a2013e871c53e8"}
Nov 03 19:50:07.072309 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:07.072253    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" podStartSLOduration=2.072234115 podStartE2EDuration="2.072234115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:50:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:50:07.067223538 +0000 UTC m=+1407.975245511" watchObservedRunningTime="2026-11-03 19:50:07.072234115 +0000 UTC m=+1407.980256057"
Nov 03 19:50:07.859567 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:07.859535    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:07.866818 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:07.866794    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:08.080880 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:08.080848    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:08.094077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:08.093785    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:50:08.302157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:08.292540    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"]
Nov 03 19:50:08.302157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:08.292793    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy" containerID="cri-o://0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7" gracePeriod=30
Nov 03 19:50:11.035634 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.035603    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"]
Nov 03 19:50:11.036363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.036348    5051 cpu_manager.go:414] "containerMap: removing container" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy"
Nov 03 19:50:11.036424 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.036365    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd2e2038-3d81-4246-8ef8-c29bfb7e8942" containerName="istio-proxy"
Nov 03 19:50:11.039521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.039504    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:50:11.050863 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.050759    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-9ce34d-dockercfg-jjm5n\""
Nov 03 19:50:11.050863 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.050816    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-9ce34d\""
Nov 03 19:50:11.057468 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.057440    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"]
Nov 03 19:50:11.081712 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.081687    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"]
Nov 03 19:50:11.084934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.084914    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:50:11.111061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.098136    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"]
Nov 03 19:50:11.326357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.325760    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnj2j\" (UniqueName: \"kubernetes.io/projected/f742538a-ef9d-48d6-a4c6-db2d7c311797-kube-api-access-lnj2j\") pod \"payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf\" (UID: \"f742538a-ef9d-48d6-a4c6-db2d7c311797\") " pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:50:11.326357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.325881    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f742538a-ef9d-48d6-a4c6-db2d7c311797-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf\" (UID: \"f742538a-ef9d-48d6-a4c6-db2d7c311797\") " pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:50:11.326357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.326209    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b180c7-1e40-49fd-81a0-c73f93567bbc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld\" (UID: \"36b180c7-1e40-49fd-81a0-c73f93567bbc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:50:11.326357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.326285    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6hp5\" (UniqueName: \"kubernetes.io/projected/36b180c7-1e40-49fd-81a0-c73f93567bbc-kube-api-access-w6hp5\") pod \"payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld\" (UID: \"36b180c7-1e40-49fd-81a0-c73f93567bbc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:50:11.427850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.427534    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b180c7-1e40-49fd-81a0-c73f93567bbc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld\" (UID: \"36b180c7-1e40-49fd-81a0-c73f93567bbc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:50:11.427850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.427639    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6hp5\" (UniqueName: \"kubernetes.io/projected/36b180c7-1e40-49fd-81a0-c73f93567bbc-kube-api-access-w6hp5\") pod \"payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld\" (UID: \"36b180c7-1e40-49fd-81a0-c73f93567bbc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:50:11.427850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.427720    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnj2j\" (UniqueName: \"kubernetes.io/projected/f742538a-ef9d-48d6-a4c6-db2d7c311797-kube-api-access-lnj2j\") pod \"payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf\" (UID: \"f742538a-ef9d-48d6-a4c6-db2d7c311797\") " pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:50:11.427850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.427848    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f742538a-ef9d-48d6-a4c6-db2d7c311797-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf\" (UID: \"f742538a-ef9d-48d6-a4c6-db2d7c311797\") " pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:50:11.428507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.428480    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b180c7-1e40-49fd-81a0-c73f93567bbc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld\" (UID: \"36b180c7-1e40-49fd-81a0-c73f93567bbc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:50:11.428654 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.428634    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f742538a-ef9d-48d6-a4c6-db2d7c311797-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf\" (UID: \"f742538a-ef9d-48d6-a4c6-db2d7c311797\") " pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:50:11.440308 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.440282    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnj2j\" (UniqueName: \"kubernetes.io/projected/f742538a-ef9d-48d6-a4c6-db2d7c311797-kube-api-access-lnj2j\") pod \"payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf\" (UID: \"f742538a-ef9d-48d6-a4c6-db2d7c311797\") " pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:50:11.440450 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.440353    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6hp5\" (UniqueName: \"kubernetes.io/projected/36b180c7-1e40-49fd-81a0-c73f93567bbc-kube-api-access-w6hp5\") pod \"payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld\" (UID: \"36b180c7-1e40-49fd-81a0-c73f93567bbc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:50:11.455520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.455485    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:50:11.655619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.655586    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:50:11.814529 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:50:11.814474    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf742538a_ef9d_48d6_a4c6_db2d7c311797.slice/crio-b0fc7c928670e778137614ad880072c22de5acc28ebc5e9a95565f783ba1b888 WatchSource:0}: Error finding container b0fc7c928670e778137614ad880072c22de5acc28ebc5e9a95565f783ba1b888: Status 404 returned error can't find the container with id b0fc7c928670e778137614ad880072c22de5acc28ebc5e9a95565f783ba1b888
Nov 03 19:50:11.816997 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.816893    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"]
Nov 03 19:50:11.970359 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:11.970330    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"]
Nov 03 19:50:11.970686 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:50:11.970664    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b180c7_1e40_49fd_81a0_c73f93567bbc.slice/crio-92dad4097df6c9f327e570a7a0095fe21961d6a90fae1caa7ed5b72eadc945b3 WatchSource:0}: Error finding container 92dad4097df6c9f327e570a7a0095fe21961d6a90fae1caa7ed5b72eadc945b3: Status 404 returned error can't find the container with id 92dad4097df6c9f327e570a7a0095fe21961d6a90fae1caa7ed5b72eadc945b3
Nov 03 19:50:12.152953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.152867    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"]
Nov 03 19:50:12.157737 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.157709    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.173125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.173097    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"]
Nov 03 19:50:12.232646 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.232611    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ab584a-openshift-default-dockercfg-h8whc\""
Nov 03 19:50:12.259709 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.259680    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-socket\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.259825 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.259785    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-certs\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.259825 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.259815    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4761a958-3fed-4c9f-aa46-314065cebb4a-istiod-ca-cert\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.260026 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.260003    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-data\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.260087 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.260059    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-token\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.260122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.260113    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-envoy\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.260160 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.260140    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsfwq\" (UniqueName: \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-kube-api-access-jsfwq\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.260191 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.260171    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-podinfo\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.260360 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.260340    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-credential-socket\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.331877 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.331841    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf" event={"ID":"f742538a-ef9d-48d6-a4c6-db2d7c311797","Type":"ContainerStarted","Data":"358006e1ab3389ee0485d5910000b3fad38fb6940441edc73a03c8f9c39be243"}
Nov 03 19:50:12.332020 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.331905    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:50:12.332020 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.331930    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf" event={"ID":"f742538a-ef9d-48d6-a4c6-db2d7c311797","Type":"ContainerStarted","Data":"b0fc7c928670e778137614ad880072c22de5acc28ebc5e9a95565f783ba1b888"}
Nov 03 19:50:12.333784 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.333757    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld" event={"ID":"36b180c7-1e40-49fd-81a0-c73f93567bbc","Type":"ContainerStarted","Data":"19cbe5c8d48fd663b25b0672c641fef0b8e25e2397f3c2465c779b55fb50d58c"}
Nov 03 19:50:12.333894 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.333805    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld" event={"ID":"36b180c7-1e40-49fd-81a0-c73f93567bbc","Type":"ContainerStarted","Data":"92dad4097df6c9f327e570a7a0095fe21961d6a90fae1caa7ed5b72eadc945b3"}
Nov 03 19:50:12.333894 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.333840    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:50:12.366063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.362418    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-credential-socket\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.366063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.362573    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-socket\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.366063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.362690    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-certs\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.366063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.362723    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4761a958-3fed-4c9f-aa46-314065cebb4a-istiod-ca-cert\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.366063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.365643    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-credential-socket\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.366451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.366156    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-data\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.366451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.366261    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-token\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.366451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.366307    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-envoy\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.366451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.366349    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsfwq\" (UniqueName: \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-kube-api-access-jsfwq\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.366779 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.366462    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-podinfo\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.368923 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.368892    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-data\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.369092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.369018    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4761a958-3fed-4c9f-aa46-314065cebb4a-istiod-ca-cert\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.369317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.369291    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-certs\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.369610 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.369586    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-socket\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.372741 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.372715    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-podinfo\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.378633 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.378355    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-envoy\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.384485 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.384454    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsfwq\" (UniqueName: \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-kube-api-access-jsfwq\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.387741 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.387697    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-token\") pod \"e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") " pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.399492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.399441    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld" podStartSLOduration=1.399426465 podStartE2EDuration="1.399426465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:50:12.393679545 +0000 UTC m=+1413.301701485" watchObservedRunningTime="2026-11-03 19:50:12.399426465 +0000 UTC m=+1413.307448404"
Nov 03 19:50:12.400476 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.400435    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf" podStartSLOduration=1.400423285 podStartE2EDuration="1.400423285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:50:12.369902071 +0000 UTC m=+1413.277924010" watchObservedRunningTime="2026-11-03 19:50:12.400423285 +0000 UTC m=+1413.308445257"
Nov 03 19:50:12.478569 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.478540    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:12.788555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.788527    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"]
Nov 03 19:50:12.788739 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:50:12.788716    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4761a958_3fed_4c9f_aa46_314065cebb4a.slice/crio-63ea5dde67aa3bde02c8789047d08cc2324efddf1029f753f81b3ef10f4c1f03 WatchSource:0}: Error finding container 63ea5dde67aa3bde02c8789047d08cc2324efddf1029f753f81b3ef10f4c1f03: Status 404 returned error can't find the container with id 63ea5dde67aa3bde02c8789047d08cc2324efddf1029f753f81b3ef10f4c1f03
Nov 03 19:50:12.792874 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.792843    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:12.792929 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.792906    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:12.792971 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:12.792937    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:13.453456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.453412    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" event={"ID":"4761a958-3fed-4c9f-aa46-314065cebb4a","Type":"ContainerStarted","Data":"a1e47d98d2349afc93b60d10631ea08a066de639c970ee387ce5c2f459482c00"}
Nov 03 19:50:13.453886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.453472    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" event={"ID":"4761a958-3fed-4c9f-aa46-314065cebb4a","Type":"ContainerStarted","Data":"63ea5dde67aa3bde02c8789047d08cc2324efddf1029f753f81b3ef10f4c1f03"}
Nov 03 19:50:13.479057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.479013    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:13.490462 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.490139    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:13.515419 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.515359    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" podStartSLOduration=1.515339295 podStartE2EDuration="1.515339295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:50:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:50:13.484288602 +0000 UTC m=+1414.392310565" watchObservedRunningTime="2026-11-03 19:50:13.515339295 +0000 UTC m=+1414.423361236"
Nov 03 19:50:13.670426 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.670395    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:50:13.758244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.758211    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-envoy\") pod \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") "
Nov 03 19:50:13.758403 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.758311    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-data\") pod \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") "
Nov 03 19:50:13.758403 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.758380    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-podinfo\") pod \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") "
Nov 03 19:50:13.758498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.758414    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-token\" (UniqueName: \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-token\") pod \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") "
Nov 03 19:50:13.758542 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.758515    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istiod-ca-cert\") pod \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") "
Nov 03 19:50:13.758597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.758558    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-credential-socket\") pod \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") "
Nov 03 19:50:13.758726 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.758707    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-certs\") pod \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") "
Nov 03 19:50:13.758809 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.758792    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-kube-api-access-xqnvd\" (UniqueName: \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-kube-api-access-xqnvd\") pod \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") "
Nov 03 19:50:13.758970 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.758953    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-socket\") pod \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\" (UID: \"ffbc34cd-6a8c-4c45-b201-d3c5de936594\") "
Nov 03 19:50:13.759342 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.759315    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-data" pod "ffbc34cd-6a8c-4c45-b201-d3c5de936594" (UID: "ffbc34cd-6a8c-4c45-b201-d3c5de936594"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.759664    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-certs" pod "ffbc34cd-6a8c-4c45-b201-d3c5de936594" (UID: "ffbc34cd-6a8c-4c45-b201-d3c5de936594"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.759973    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istiod-ca-cert" pod "ffbc34cd-6a8c-4c45-b201-d3c5de936594" (UID: "ffbc34cd-6a8c-4c45-b201-d3c5de936594"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.760657    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-socket" pod "ffbc34cd-6a8c-4c45-b201-d3c5de936594" (UID: "ffbc34cd-6a8c-4c45-b201-d3c5de936594"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.761291    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-podinfo" pod "ffbc34cd-6a8c-4c45-b201-d3c5de936594" (UID: "ffbc34cd-6a8c-4c45-b201-d3c5de936594"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.762153    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-token" pod "ffbc34cd-6a8c-4c45-b201-d3c5de936594" (UID: "ffbc34cd-6a8c-4c45-b201-d3c5de936594"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.762490    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-envoy" pod "ffbc34cd-6a8c-4c45-b201-d3c5de936594" (UID: "ffbc34cd-6a8c-4c45-b201-d3c5de936594"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.763262    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.763290    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.763306    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.763319    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.763332    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.763346    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.763361    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:13.763507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.763462    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-credential-socket" pod "ffbc34cd-6a8c-4c45-b201-d3c5de936594" (UID: "ffbc34cd-6a8c-4c45-b201-d3c5de936594"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:13.766405 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.766373    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-kube-api-access-xqnvd" pod "ffbc34cd-6a8c-4c45-b201-d3c5de936594" (UID: "ffbc34cd-6a8c-4c45-b201-d3c5de936594"). InnerVolumeSpecName "kube-api-access-xqnvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:13.865709 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.865672    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffbc34cd-6a8c-4c45-b201-d3c5de936594-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:13.865709 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:13.865709    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqnvd\" (UniqueName: \"kubernetes.io/projected/ffbc34cd-6a8c-4c45-b201-d3c5de936594-kube-api-access-xqnvd\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:14.460760 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.460719    5051 generic.go:356] "Generic (PLEG): container finished" podID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerID="0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7" exitCode=0
Nov 03 19:50:14.461238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.460793    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"
Nov 03 19:50:14.461238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.460808    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" event={"ID":"ffbc34cd-6a8c-4c45-b201-d3c5de936594","Type":"ContainerDied","Data":"0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7"}
Nov 03 19:50:14.461238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.460866    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p" event={"ID":"ffbc34cd-6a8c-4c45-b201-d3c5de936594","Type":"ContainerDied","Data":"f826e95ccf45406e01f8df98d83679b5cb7049ffdecef8c0619a84380fc92232"}
Nov 03 19:50:14.461238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.460890    5051 scope.go:122] "RemoveContainer" containerID="0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7"
Nov 03 19:50:14.461462 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.461397    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:14.462640 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.462618    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:50:14.473228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.473206    5051 scope.go:122] "RemoveContainer" containerID="e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d"
Nov 03 19:50:14.483456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.483438    5051 scope.go:122] "RemoveContainer" containerID="0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7"
Nov 03 19:50:14.483754 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:14.483726    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7\": container with ID starting with 0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7 not found: ID does not exist" containerID="0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7"
Nov 03 19:50:14.483800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.483764    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7"} err="failed to get container status \"0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7\": rpc error: code = NotFound desc = could not find container \"0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7\": container with ID starting with 0a09661507895e814986e0246e9e3352279ddf422bfaf7f9473f233f345b73b7 not found: ID does not exist"
Nov 03 19:50:14.483800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.483782    5051 scope.go:122] "RemoveContainer" containerID="e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d"
Nov 03 19:50:14.484032 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:14.484014    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d\": container with ID starting with e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d not found: ID does not exist" containerID="e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d"
Nov 03 19:50:14.484083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.484036    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d"} err="failed to get container status \"e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d\": rpc error: code = NotFound desc = could not find container \"e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d\": container with ID starting with e9fdc88f2e5c17847aa0a9bb1bf4b782a34f1f458c852653679783fa5058d01d not found: ID does not exist"
Nov 03 19:50:14.516743 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.516714    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"]
Nov 03 19:50:14.521723 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:14.521695    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-eae28ff0-openshift-default-5f789b85-m7z6p"]
Nov 03 19:50:15.231236 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:15.231205    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" path="/var/lib/kubelet/pods/ffbc34cd-6a8c-4c45-b201-d3c5de936594/volumes"
Nov 03 19:50:16.332030 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.331999    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"]
Nov 03 19:50:16.332745 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.332729    5051 cpu_manager.go:414] "containerMap: removing container" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy"
Nov 03 19:50:16.332791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.332746    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy"
Nov 03 19:50:16.332908 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.332899    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy"
Nov 03 19:50:16.332941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.332919    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy"
Nov 03 19:50:16.336394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.336368    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.352737 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.352712    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-5444b0-serving-cert\""
Nov 03 19:50:16.359494 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.358940    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"]
Nov 03 19:50:16.404920 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.404886    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.405133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.405111    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwf8t\" (UniqueName: \"kubernetes.io/projected/3843274d-f1ec-4da5-806f-e841042b92f6-kube-api-access-hwf8t\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.405191 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.405175    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.405287 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.405270    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3843274d-f1ec-4da5-806f-e841042b92f6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.507843 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.507806    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.512212 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.512170    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3843274d-f1ec-4da5-806f-e841042b92f6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.512428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.512407    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.512613 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.512590    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwf8t\" (UniqueName: \"kubernetes.io/projected/3843274d-f1ec-4da5-806f-e841042b92f6-kube-api-access-hwf8t\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.514699 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.514258    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3843274d-f1ec-4da5-806f-e841042b92f6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.520768 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.520714    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.521001 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.520809    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.537006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.528994    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwf8t\" (UniqueName: \"kubernetes.io/projected/3843274d-f1ec-4da5-806f-e841042b92f6-kube-api-access-hwf8t\") pod \"maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.663361 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.663282    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:16.951024 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:50:16.950989    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3843274d_f1ec_4da5_806f_e841042b92f6.slice/crio-9b65edaceb15a39a26d231c727c41139dc92f9ad6e54cd00a68f77bdb0188343 WatchSource:0}: Error finding container 9b65edaceb15a39a26d231c727c41139dc92f9ad6e54cd00a68f77bdb0188343: Status 404 returned error can't find the container with id 9b65edaceb15a39a26d231c727c41139dc92f9ad6e54cd00a68f77bdb0188343
Nov 03 19:50:16.951237 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:16.951214    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"]
Nov 03 19:50:17.502086 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.496549    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk" event={"ID":"3843274d-f1ec-4da5-806f-e841042b92f6","Type":"ContainerStarted","Data":"9b65edaceb15a39a26d231c727c41139dc92f9ad6e54cd00a68f77bdb0188343"}
Nov 03 19:50:17.587528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.579529    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"]
Nov 03 19:50:17.587528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.581494    5051 cpu_manager.go:414] "containerMap: removing container" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy"
Nov 03 19:50:17.587528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.581516    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffbc34cd-6a8c-4c45-b201-d3c5de936594" containerName="istio-proxy"
Nov 03 19:50:17.587528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.587197    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:50:17.596276 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.596243    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-ab584a\""
Nov 03 19:50:17.596417 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.596332    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-ab584a-dockercfg-mz46b\""
Nov 03 19:50:17.601940 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.601437    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"]
Nov 03 19:50:17.754453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.754405    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w\" (UID: \"0e1ad4ab-300a-425c-b752-bd6dd6108c4a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:50:17.754655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.754561    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djzzt\" (UniqueName: \"kubernetes.io/projected/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-kube-api-access-djzzt\") pod \"payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w\" (UID: \"0e1ad4ab-300a-425c-b752-bd6dd6108c4a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:50:17.856259 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.856226    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w\" (UID: \"0e1ad4ab-300a-425c-b752-bd6dd6108c4a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:50:17.856416 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.856347    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djzzt\" (UniqueName: \"kubernetes.io/projected/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-kube-api-access-djzzt\") pod \"payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w\" (UID: \"0e1ad4ab-300a-425c-b752-bd6dd6108c4a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:50:17.857009 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.856989    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w\" (UID: \"0e1ad4ab-300a-425c-b752-bd6dd6108c4a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:50:17.869662 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.869629    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djzzt\" (UniqueName: \"kubernetes.io/projected/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-kube-api-access-djzzt\") pod \"payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w\" (UID: \"0e1ad4ab-300a-425c-b752-bd6dd6108c4a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:50:17.915375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:17.915345    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:50:18.291062 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:50:18.274593    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1ad4ab_300a_425c_b752_bd6dd6108c4a.slice/crio-81991c7e00442b50a9f45c02a7d76c65dbf5bbf6c36c2794a10d7edb6a4a5403 WatchSource:0}: Error finding container 81991c7e00442b50a9f45c02a7d76c65dbf5bbf6c36c2794a10d7edb6a4a5403: Status 404 returned error can't find the container with id 81991c7e00442b50a9f45c02a7d76c65dbf5bbf6c36c2794a10d7edb6a4a5403
Nov 03 19:50:18.295091 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:18.291721    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"]
Nov 03 19:50:18.506340 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:18.506298    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk" event={"ID":"3843274d-f1ec-4da5-806f-e841042b92f6","Type":"ContainerStarted","Data":"441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e"}
Nov 03 19:50:18.506859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:18.506371    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:18.507974 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:18.507932    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w" event={"ID":"0e1ad4ab-300a-425c-b752-bd6dd6108c4a","Type":"ContainerStarted","Data":"ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5"}
Nov 03 19:50:18.507974 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:18.507973    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w" event={"ID":"0e1ad4ab-300a-425c-b752-bd6dd6108c4a","Type":"ContainerStarted","Data":"81991c7e00442b50a9f45c02a7d76c65dbf5bbf6c36c2794a10d7edb6a4a5403"}
Nov 03 19:50:18.508165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:18.508014    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:50:18.531373 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:18.531269    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk" podStartSLOduration=2.033274324 podStartE2EDuration="2.53125057s" totalImagesPullingTime="497.976246ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:50:18.528736773 +0000 UTC m=+1419.436758735" watchObservedRunningTime="2026-11-03 19:50:18.53125057 +0000 UTC m=+1419.439272721"
Nov 03 19:50:18.554439 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:18.554381    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w" podStartSLOduration=1.5543629079999999 podStartE2EDuration="1.554362908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:50:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:50:18.552545797 +0000 UTC m=+1419.460567755" watchObservedRunningTime="2026-11-03 19:50:18.554362908 +0000 UTC m=+1419.462384847"
Nov 03 19:50:20.389579 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:20.389545    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:50:23.494309 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:23.494266    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:50:23.494845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:23.494349    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:50:24.546070 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:24.543857    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:50:26.602063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:26.601842    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"]
Nov 03 19:50:26.620693 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:26.619513    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"]
Nov 03 19:50:26.620693 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:26.619631    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:26.624090 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:26.624034    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-9ce34d-serving-cert\""
Nov 03 19:50:26.920331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:26.920250    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-metrics-certs\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:26.920331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:26.920300    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hblh\" (UniqueName: \"kubernetes.io/projected/1cf13e8c-1976-4218-943b-4deaf2dde7eb-kube-api-access-7hblh\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:26.920331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:26.920317    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-maas-api-tls\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:26.920550 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:26.920394    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cf13e8c-1976-4218-943b-4deaf2dde7eb-openshift-service-ca\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:27.021857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:27.021821    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-metrics-certs\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:27.022027 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:27.021872    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hblh\" (UniqueName: \"kubernetes.io/projected/1cf13e8c-1976-4218-943b-4deaf2dde7eb-kube-api-access-7hblh\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:27.022027 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:27.021918    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-maas-api-tls\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:27.022138 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:27.022028    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cf13e8c-1976-4218-943b-4deaf2dde7eb-openshift-service-ca\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:27.023000 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:27.022973    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cf13e8c-1976-4218-943b-4deaf2dde7eb-openshift-service-ca\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:27.024596 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:27.024574    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-maas-api-tls\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:27.024670 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:27.024609    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-metrics-certs\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:27.034631 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:27.034605    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hblh\" (UniqueName: \"kubernetes.io/projected/1cf13e8c-1976-4218-943b-4deaf2dde7eb-kube-api-access-7hblh\") pod \"maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:27.140066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:27.135509    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:27.704080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:27.704053    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"]
Nov 03 19:50:27.923064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:27.916948    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf" event={"ID":"1cf13e8c-1976-4218-943b-4deaf2dde7eb","Type":"ContainerStarted","Data":"7cabc386825c2979d317dc6276c981fc4e43816f663b3a677906b5c00ddbf38d"}
Nov 03 19:50:28.926422 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:28.926377    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf" event={"ID":"1cf13e8c-1976-4218-943b-4deaf2dde7eb","Type":"ContainerStarted","Data":"6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2"}
Nov 03 19:50:28.926808 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:28.926494    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:28.951621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:28.951570    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf" podStartSLOduration=2.457191037 podStartE2EDuration="2.951556889s" totalImagesPullingTime="494.365852ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:50:28.946536037 +0000 UTC m=+1429.854557989" watchObservedRunningTime="2026-11-03 19:50:28.951556889 +0000 UTC m=+1429.859578829"
Nov 03 19:50:29.521488 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:29.521439    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:50:31.240611 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.240574    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"]
Nov 03 19:50:31.246693 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.246669    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.256770 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.256744    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"]
Nov 03 19:50:31.257804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.257776    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ab584a-serving-cert\""
Nov 03 19:50:31.362514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.361793    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c49123a-fa09-4756-8083-6d3a5c4d3f48-openshift-service-ca\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.362514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.361939    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-maas-api-tls\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.362514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.361979    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s447d\" (UniqueName: \"kubernetes.io/projected/8c49123a-fa09-4756-8083-6d3a5c4d3f48-kube-api-access-s447d\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.362514 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.362063    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-metrics-certs\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.392828 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.392796    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"]
Nov 03 19:50:31.407728 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.407701    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.416456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.416377    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"]
Nov 03 19:50:31.467310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.467268    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-metrics-certs\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.467577 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.467556    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-envoy\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.467799 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.467750    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.467935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.467919    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-certs\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.470020 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.469996    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-data\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.470353 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.470333    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-credential-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.470503 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.470487    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-podinfo\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.470855 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.470833    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bpwg\" (UniqueName: \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-kube-api-access-4bpwg\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.471186 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.471167    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istiod-ca-cert\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.471358 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.471342    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c49123a-fa09-4756-8083-6d3a5c4d3f48-openshift-service-ca\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.471484 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.471470    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-token\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.471686 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.471673    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-maas-api-tls\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.471803 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.471789    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s447d\" (UniqueName: \"kubernetes.io/projected/8c49123a-fa09-4756-8083-6d3a5c4d3f48-kube-api-access-s447d\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.473232 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.473208    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c49123a-fa09-4756-8083-6d3a5c4d3f48-openshift-service-ca\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.480764 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.480733    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-metrics-certs\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.486375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.486344    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s447d\" (UniqueName: \"kubernetes.io/projected/8c49123a-fa09-4756-8083-6d3a5c4d3f48-kube-api-access-s447d\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.490375 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.490346    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-maas-api-tls\") pod \"maas-api-e2e-api-b-ab584a-598d98656c-dp9r2\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.572867 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.572774    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-envoy\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.572867 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.572845    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.573165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.572871    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-certs\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.573165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.572924    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-data\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.573165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.572952    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-credential-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.573165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.572985    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-podinfo\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.573165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.573079    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bpwg\" (UniqueName: \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-kube-api-access-4bpwg\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.573423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.573235    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istiod-ca-cert\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.573423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.573278    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.573423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.573311    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-token\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.573423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.573371    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-data\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.573617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.573526    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-certs\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.573783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.573761    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-credential-socket\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.574098 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.574013    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istiod-ca-cert\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.576283 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.576255    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-podinfo\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.580280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.580254    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-envoy\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.586366 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.586169    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:31.586494 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.586461    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bpwg\" (UniqueName: \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-kube-api-access-4bpwg\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.587317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.587295    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-token\") pod \"e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") " pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.776307 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.776261    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:31.913052 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:31.912999    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"]
Nov 03 19:50:32.072594 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:32.072558    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2" event={"ID":"8c49123a-fa09-4756-8083-6d3a5c4d3f48","Type":"ContainerStarted","Data":"ad11f5851d47409a9bab04bac3c505995e723c4a5feb13ae4c7229344d00c0bc"}
Nov 03 19:50:32.124555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:32.123933    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:32.124555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:32.124027    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:32.124555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:32.124091    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:32.131598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:32.129920    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"]
Nov 03 19:50:32.780609 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:32.780518    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:50:33.093059 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:33.093012    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2" event={"ID":"8c49123a-fa09-4756-8083-6d3a5c4d3f48","Type":"ContainerStarted","Data":"7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1"}
Nov 03 19:50:33.094052 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:33.094024    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:33.095988 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:33.095960    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj" event={"ID":"965226d2-5ae1-4bdf-97b3-6726e9c035d7","Type":"ContainerStarted","Data":"9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7"}
Nov 03 19:50:33.096134 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:33.096015    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj" event={"ID":"965226d2-5ae1-4bdf-97b3-6726e9c035d7","Type":"ContainerStarted","Data":"ee4a3282a95e56499b7d54eb3b07810b8b8080ec90d1e763be04be1c28447f84"}
Nov 03 19:50:33.122335 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:33.122286    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2" podStartSLOduration=1.6426510699999999 podStartE2EDuration="2.122269688s" totalImagesPullingTime="479.618618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:50:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:50:33.115382055 +0000 UTC m=+1434.023403995" watchObservedRunningTime="2026-11-03 19:50:33.122269688 +0000 UTC m=+1434.030291630"
Nov 03 19:50:33.127381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:33.126983    5051 patch_prober.go:29] interesting pod/e2e-api-a-300831-openshift-default-656d95d487-xtm2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": read tcp 10.132.0.2:41526->10.132.0.121:15021: read: connection reset by peer" start-of-body=
Nov 03 19:50:33.127381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:33.127036    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": read tcp 10.132.0.2:41526->10.132.0.121:15021: read: connection reset by peer"
Nov 03 19:50:33.160119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:33.155780    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj" podStartSLOduration=2.155758626 podStartE2EDuration="2.155758626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:50:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:50:33.149216025 +0000 UTC m=+1434.057237977" watchObservedRunningTime="2026-11-03 19:50:33.155758626 +0000 UTC m=+1434.063780566"
Nov 03 19:50:33.776805 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:33.776775    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:33.778216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:33.778192    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused" start-of-body=
Nov 03 19:50:33.778293 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:33.778243    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj" podUID="965226d2-5ae1-4bdf-97b3-6726e9c035d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused"
Nov 03 19:50:34.106267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.106187    5051 generic.go:356] "Generic (PLEG): container finished" podID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerID="96d8f4c3450d20f32bb5000ef953f7fd9e9e66ec16cb4849373169f4faebf089" exitCode=0
Nov 03 19:50:34.106717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.106264    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" event={"ID":"425fcd7d-1420-41a8-89d4-503a9d259b89","Type":"ContainerDied","Data":"96d8f4c3450d20f32bb5000ef953f7fd9e9e66ec16cb4849373169f4faebf089"}
Nov 03 19:50:34.106801 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.106728    5051 scope.go:122] "RemoveContainer" containerID="96d8f4c3450d20f32bb5000ef953f7fd9e9e66ec16cb4849373169f4faebf089"
Nov 03 19:50:34.108240 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.108212    5051 generic.go:356] "Generic (PLEG): container finished" podID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerID="eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841" exitCode=0
Nov 03 19:50:34.108349 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.108288    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" event={"ID":"082b8718-d755-4a33-9cc5-f1a157b846ec","Type":"ContainerDied","Data":"eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841"}
Nov 03 19:50:34.108693 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.108659    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:34.108793 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.108751    5051 scope.go:122] "RemoveContainer" containerID="eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841"
Nov 03 19:50:34.108793 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.108751    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:34.108906 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.108860    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:34.110426 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.110400    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:34.110521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.110459    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:34.110521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.110494    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:34.698356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.698325    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:50:34.777011 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.776808    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused" start-of-body=
Nov 03 19:50:34.777011 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.776908    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj" podUID="965226d2-5ae1-4bdf-97b3-6726e9c035d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused"
Nov 03 19:50:34.965572 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:34.965544    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:50:35.120791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:35.120753    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" event={"ID":"082b8718-d755-4a33-9cc5-f1a157b846ec","Type":"ContainerStarted","Data":"7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06"}
Nov 03 19:50:35.123700 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:35.123674    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" event={"ID":"425fcd7d-1420-41a8-89d4-503a9d259b89","Type":"ContainerStarted","Data":"c1442e341e4aaff70f0a0f757c955e45a6996fa5901e0b3ab4c883b1e5e23867"}
Nov 03 19:50:35.335715 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:35.333810    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:50:35.335715 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:35.333931    5051 patch_prober.go:29] interesting pod/e2e-api-a-300831-openshift-default-656d95d487-xtm2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 03 19:50:35.335715 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:35.333975    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 03 19:50:35.711396 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:35.711362    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:50:35.711798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:35.711773    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 19:50:35.711987 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:35.711964    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 19:50:35.782465 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:35.782429    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused" start-of-body=
Nov 03 19:50:35.782755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:35.782725    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj" podUID="965226d2-5ae1-4bdf-97b3-6726e9c035d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused"
Nov 03 19:50:36.324262 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:36.324210    5051 patch_prober.go:29] interesting pod/e2e-api-a-300831-openshift-default-656d95d487-xtm2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 03 19:50:36.324683 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:36.324282    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 03 19:50:36.721066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:36.720155    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 19:50:36.721066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:36.720228    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 19:50:36.787019 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:36.786979    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused" start-of-body=
Nov 03 19:50:36.787311 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:36.787279    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj" podUID="965226d2-5ae1-4bdf-97b3-6726e9c035d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused"
Nov 03 19:50:37.324709 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:37.324664    5051 patch_prober.go:29] interesting pod/e2e-api-a-300831-openshift-default-656d95d487-xtm2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 03 19:50:37.325310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:37.324722    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 03 19:50:37.725125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:37.714207    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 19:50:37.725125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:37.714284    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 19:50:37.789067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:37.780534    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused" start-of-body=
Nov 03 19:50:37.789067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:37.780619    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj" podUID="965226d2-5ae1-4bdf-97b3-6726e9c035d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused"
Nov 03 19:50:38.324504 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:38.324462    5051 patch_prober.go:29] interesting pod/e2e-api-a-300831-openshift-default-656d95d487-xtm2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 03 19:50:38.324688 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:38.324529    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 03 19:50:38.733124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:38.732467    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 19:50:38.733124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:38.732528    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 19:50:38.904392 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:38.904362    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:38.906856 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:38.906825    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:38.910615 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:38.910546    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:50:39.324000 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:39.323955    5051 patch_prober.go:29] interesting pod/e2e-api-a-300831-openshift-default-656d95d487-xtm2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 03 19:50:39.324195 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:39.324012    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 03 19:50:39.710747 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:39.710722    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 19:50:39.710912 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:39.710768    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 19:50:40.129094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:40.129013    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:50:40.324152 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:40.324111    5051 patch_prober.go:29] interesting pod/e2e-api-a-300831-openshift-default-656d95d487-xtm2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 03 19:50:40.324342 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:40.324168    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 03 19:50:40.717160 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:40.717119    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 19:50:40.717443 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:40.717407    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 19:50:40.908543 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:40.908491    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"]
Nov 03 19:50:40.908809 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:40.908764    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" podUID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" containerName="istio-proxy" containerID="cri-o://845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d" gracePeriod=30
Nov 03 19:50:41.329518 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:41.329489    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:50:41.330020 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:41.330002    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:50:41.330659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:41.330642    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:50:41.716120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:41.716094    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:50:41.716625 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:41.716605    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:50:41.717445 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:41.717427    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:50:42.730399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:42.730351    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"]
Nov 03 19:50:42.731443 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:42.730604    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz" podUID="245556da-83c7-4339-8d4b-9be9c738a8e0" containerName="maas-api" containerID="cri-o://f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c" gracePeriod=30
Nov 03 19:50:43.187053 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.187011    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:50:43.279623 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.279593    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/245556da-83c7-4339-8d4b-9be9c738a8e0-kube-api-access-ks77k\" (UniqueName: \"kubernetes.io/projected/245556da-83c7-4339-8d4b-9be9c738a8e0-kube-api-access-ks77k\") pod \"245556da-83c7-4339-8d4b-9be9c738a8e0\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") "
Nov 03 19:50:43.279844 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.279686    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/245556da-83c7-4339-8d4b-9be9c738a8e0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245556da-83c7-4339-8d4b-9be9c738a8e0-openshift-service-ca\") pod \"245556da-83c7-4339-8d4b-9be9c738a8e0\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") "
Nov 03 19:50:43.279844 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.279742    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-metrics-certs\") pod \"245556da-83c7-4339-8d4b-9be9c738a8e0\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") "
Nov 03 19:50:43.279844 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.279773    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-maas-api-tls\") pod \"245556da-83c7-4339-8d4b-9be9c738a8e0\" (UID: \"245556da-83c7-4339-8d4b-9be9c738a8e0\") "
Nov 03 19:50:43.280204 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.280157    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/245556da-83c7-4339-8d4b-9be9c738a8e0-openshift-service-ca" pod "245556da-83c7-4339-8d4b-9be9c738a8e0" (UID: "245556da-83c7-4339-8d4b-9be9c738a8e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:43.280578 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.280556    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245556da-83c7-4339-8d4b-9be9c738a8e0-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:43.282034 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.282012    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-metrics-certs" pod "245556da-83c7-4339-8d4b-9be9c738a8e0" (UID: "245556da-83c7-4339-8d4b-9be9c738a8e0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:50:43.282374 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.282350    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-maas-api-tls" pod "245556da-83c7-4339-8d4b-9be9c738a8e0" (UID: "245556da-83c7-4339-8d4b-9be9c738a8e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:50:43.282469 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.282409    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/245556da-83c7-4339-8d4b-9be9c738a8e0-kube-api-access-ks77k" pod "245556da-83c7-4339-8d4b-9be9c738a8e0" (UID: "245556da-83c7-4339-8d4b-9be9c738a8e0"). InnerVolumeSpecName "kube-api-access-ks77k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:43.344647 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.344618    5051 generic.go:356] "Generic (PLEG): container finished" podID="245556da-83c7-4339-8d4b-9be9c738a8e0" containerID="f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c" exitCode=0
Nov 03 19:50:43.344798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.344687    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"
Nov 03 19:50:43.344798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.344713    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz" event={"ID":"245556da-83c7-4339-8d4b-9be9c738a8e0","Type":"ContainerDied","Data":"f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c"}
Nov 03 19:50:43.344798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.344784    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz" event={"ID":"245556da-83c7-4339-8d4b-9be9c738a8e0","Type":"ContainerDied","Data":"54466e3128264ea9dd8ffa57efa93f6a3d40302109c22dc5b243594a9194edef"}
Nov 03 19:50:43.344953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.344818    5051 scope.go:122] "RemoveContainer" containerID="f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c"
Nov 03 19:50:43.360174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.360152    5051 scope.go:122] "RemoveContainer" containerID="f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c"
Nov 03 19:50:43.360624 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:43.360599    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c\": container with ID starting with f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c not found: ID does not exist" containerID="f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c"
Nov 03 19:50:43.360737 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.360635    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c"} err="failed to get container status \"f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c\": rpc error: code = NotFound desc = could not find container \"f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c\": container with ID starting with f714f334340e5bb70d265860db6edaf852143cb8171ab390fa92186345d3077c not found: ID does not exist"
Nov 03 19:50:43.374845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.374821    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"]
Nov 03 19:50:43.379608 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.379584    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c5f6cb-775687868b-vvcjz"]
Nov 03 19:50:43.381899 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.381876    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks77k\" (UniqueName: \"kubernetes.io/projected/245556da-83c7-4339-8d4b-9be9c738a8e0-kube-api-access-ks77k\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:43.381975 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.381900    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:43.381975 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:43.381917    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245556da-83c7-4339-8d4b-9be9c738a8e0-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:45.231501 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:45.231472    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="245556da-83c7-4339-8d4b-9be9c738a8e0" path="/var/lib/kubelet/pods/245556da-83c7-4339-8d4b-9be9c738a8e0/volumes"
Nov 03 19:50:45.637823 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:45.637727    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"]
Nov 03 19:50:45.646036 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:45.646003    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"]
Nov 03 19:50:46.211925 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.211892    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:50:46.338872 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.338843    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-token\" (UniqueName: \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-token\") pod \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") "
Nov 03 19:50:46.339385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.338896    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-data\") pod \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") "
Nov 03 19:50:46.339385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.338941    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-credential-socket\") pod \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") "
Nov 03 19:50:46.339385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.338991    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-kube-api-access-pt8fw\" (UniqueName: \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-kube-api-access-pt8fw\") pod \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") "
Nov 03 19:50:46.339385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.339014    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istiod-ca-cert\") pod \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") "
Nov 03 19:50:46.339385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.339131    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-certs\") pod \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") "
Nov 03 19:50:46.339385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.339163    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-data" pod "6b3a3536-bf26-42f1-8cb7-e3bb19602c16" (UID: "6b3a3536-bf26-42f1-8cb7-e3bb19602c16"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:46.339385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.339286    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-socket\") pod \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") "
Nov 03 19:50:46.339385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.339363    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-credential-socket" pod "6b3a3536-bf26-42f1-8cb7-e3bb19602c16" (UID: "6b3a3536-bf26-42f1-8cb7-e3bb19602c16"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:46.339936 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.339436    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-podinfo\") pod \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") "
Nov 03 19:50:46.339936 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.339483    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-certs" pod "6b3a3536-bf26-42f1-8cb7-e3bb19602c16" (UID: "6b3a3536-bf26-42f1-8cb7-e3bb19602c16"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:46.339936 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.339490    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-envoy\") pod \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\" (UID: \"6b3a3536-bf26-42f1-8cb7-e3bb19602c16\") "
Nov 03 19:50:46.339936 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.339494    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istiod-ca-cert" pod "6b3a3536-bf26-42f1-8cb7-e3bb19602c16" (UID: "6b3a3536-bf26-42f1-8cb7-e3bb19602c16"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:46.339936 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.339516    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-socket" pod "6b3a3536-bf26-42f1-8cb7-e3bb19602c16" (UID: "6b3a3536-bf26-42f1-8cb7-e3bb19602c16"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:46.341132 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.341109    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:46.341246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.341137    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:46.341246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.341153    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:46.341246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.341180    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:46.341246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.341195    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:46.341781 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.341754    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-kube-api-access-pt8fw" pod "6b3a3536-bf26-42f1-8cb7-e3bb19602c16" (UID: "6b3a3536-bf26-42f1-8cb7-e3bb19602c16"). InnerVolumeSpecName "kube-api-access-pt8fw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:46.342497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.342476    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-token" pod "6b3a3536-bf26-42f1-8cb7-e3bb19602c16" (UID: "6b3a3536-bf26-42f1-8cb7-e3bb19602c16"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:46.342677 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.342655    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-podinfo" pod "6b3a3536-bf26-42f1-8cb7-e3bb19602c16" (UID: "6b3a3536-bf26-42f1-8cb7-e3bb19602c16"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:50:46.342938 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.342919    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-envoy" pod "6b3a3536-bf26-42f1-8cb7-e3bb19602c16" (UID: "6b3a3536-bf26-42f1-8cb7-e3bb19602c16"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:46.379524 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.379499    5051 generic.go:356] "Generic (PLEG): container finished" podID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" containerID="845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d" exitCode=0
Nov 03 19:50:46.379626 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.379553    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"
Nov 03 19:50:46.379626 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.379591    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" event={"ID":"6b3a3536-bf26-42f1-8cb7-e3bb19602c16","Type":"ContainerDied","Data":"845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d"}
Nov 03 19:50:46.379703 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.379648    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns" event={"ID":"6b3a3536-bf26-42f1-8cb7-e3bb19602c16","Type":"ContainerDied","Data":"33dfd8b2bc6c1c90a6ff2b4e7bb5531f9830f638d73a34587d9206a3f3c9ba67"}
Nov 03 19:50:46.379703 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.379673    5051 scope.go:122] "RemoveContainer" containerID="845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d"
Nov 03 19:50:46.390735 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.390719    5051 scope.go:122] "RemoveContainer" containerID="845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d"
Nov 03 19:50:46.390966 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:46.390947    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d\": container with ID starting with 845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d not found: ID does not exist" containerID="845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d"
Nov 03 19:50:46.391017 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.390972    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d"} err="failed to get container status \"845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d\": rpc error: code = NotFound desc = could not find container \"845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d\": container with ID starting with 845832a32548c8ae83f713e30a336e1b2907b5ceb75999cd1319800a144dc15d not found: ID does not exist"
Nov 03 19:50:46.405242 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.405222    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"]
Nov 03 19:50:46.409310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.409290    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-m58ns"]
Nov 03 19:50:46.459119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.452657    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:46.459119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.452695    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:46.459119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.452712    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:46.459119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.452726    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt8fw\" (UniqueName: \"kubernetes.io/projected/6b3a3536-bf26-42f1-8cb7-e3bb19602c16-kube-api-access-pt8fw\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:46.602319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.597927    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"]
Nov 03 19:50:46.602319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.599244    5051 cpu_manager.go:414] "containerMap: removing container" podUID="245556da-83c7-4339-8d4b-9be9c738a8e0" containerName="maas-api"
Nov 03 19:50:46.602319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.599267    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="245556da-83c7-4339-8d4b-9be9c738a8e0" containerName="maas-api"
Nov 03 19:50:46.602319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.599305    5051 cpu_manager.go:414] "containerMap: removing container" podUID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" containerName="istio-proxy"
Nov 03 19:50:46.602319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.599313    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" containerName="istio-proxy"
Nov 03 19:50:46.602319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.599525    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" containerName="istio-proxy"
Nov 03 19:50:46.602319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.599559    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="245556da-83c7-4339-8d4b-9be9c738a8e0" containerName="maas-api"
Nov 03 19:50:46.605552 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.605532    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"]
Nov 03 19:50:46.605775 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.605763    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.612393 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.610985    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-c6a08e-openshift-default-dockercfg-z5zp9\""
Nov 03 19:50:46.759123 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.759086    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-podinfo\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.759290 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.759180    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-socket\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.759290 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.759222    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a258474-8531-448f-8e4a-b7e4b39728fc-istiod-ca-cert\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.759290 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.759241    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq6sn\" (UniqueName: \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-kube-api-access-zq6sn\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.759394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.759321    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-credential-socket\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.759461 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.759447    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-certs\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.759499 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.759476    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-envoy\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.759499 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.759495    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-data\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.759698 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.759686    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-token\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.860802 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.860769    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-token\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.860984 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.860894    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-podinfo\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.860984 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.860940    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-socket\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.861120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.860982    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a258474-8531-448f-8e4a-b7e4b39728fc-istiod-ca-cert\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.861120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.861010    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq6sn\" (UniqueName: \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-kube-api-access-zq6sn\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.861219 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.861118    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-credential-socket\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.861272 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.861244    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-certs\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.861322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.861280    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-envoy\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.861322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.861313    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-data\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.861461 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.861423    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-socket\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.861548 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.861507    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-credential-socket\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.861603 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.861570    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-certs\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.861778 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.861756    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-data\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.861847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.861793    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a258474-8531-448f-8e4a-b7e4b39728fc-istiod-ca-cert\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.863498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.863476    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-envoy\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.863603 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.863576    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-podinfo\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.875414 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.875387    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-token\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.875497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.875383    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq6sn\" (UniqueName: \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-kube-api-access-zq6sn\") pod \"e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") " pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:46.955435 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:46.955408    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:47.231024 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:47.230990    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b3a3536-bf26-42f1-8cb7-e3bb19602c16" path="/var/lib/kubelet/pods/6b3a3536-bf26-42f1-8cb7-e3bb19602c16/volumes"
Nov 03 19:50:47.263795 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:47.263770    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"]
Nov 03 19:50:47.264394 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:50:47.264369    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a258474_8531_448f_8e4a_b7e4b39728fc.slice/crio-74d3569bd0298a318ccf0afef619893f9c554c763f432d6ccf934130b76cf41d WatchSource:0}: Error finding container 74d3569bd0298a318ccf0afef619893f9c554c763f432d6ccf934130b76cf41d: Status 404 returned error can't find the container with id 74d3569bd0298a318ccf0afef619893f9c554c763f432d6ccf934130b76cf41d
Nov 03 19:50:47.267818 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:47.267788    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:47.267873 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:47.267852    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:47.267907 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:47.267886    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:50:47.393282 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:47.393244    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" event={"ID":"7a258474-8531-448f-8e4a-b7e4b39728fc","Type":"ContainerStarted","Data":"9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0"}
Nov 03 19:50:47.393657 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:47.393341    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" event={"ID":"7a258474-8531-448f-8e4a-b7e4b39728fc","Type":"ContainerStarted","Data":"74d3569bd0298a318ccf0afef619893f9c554c763f432d6ccf934130b76cf41d"}
Nov 03 19:50:47.424819 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:47.424762    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" podStartSLOduration=1.424749428 podStartE2EDuration="1.424749428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:50:47.422392532 +0000 UTC m=+1448.330414476" watchObservedRunningTime="2026-11-03 19:50:47.424749428 +0000 UTC m=+1448.332771364"
Nov 03 19:50:47.956400 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:47.956366    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:48.966554 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:48.966522    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:49.077575 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:49.075816    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"]
Nov 03 19:50:49.099087 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:49.096416    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"]
Nov 03 19:50:49.446119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:49.445020    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:49.446298 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:49.446255    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:50:51.166875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:51.166842    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"]
Nov 03 19:50:51.186309 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:51.186276    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"]
Nov 03 19:50:54.163349 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:54.105292    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh" podUID="f4ebb6dd-147d-4669-b280-2f77e334f18b" containerName="payload-pre-processing" containerID="cri-o://58b19ff9f6f396aa154ed3fa3c6410dc3fc7409fae83ab2a622b122ac6241ba2" gracePeriod=25
Nov 03 19:50:54.163349 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:54.109918    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv" podUID="eaab2337-0cb8-4c1b-8155-45ca5ebad934" containerName="payload-processing" containerID="cri-o://82a62337483b2c1c89d7004ff499e635dfcdcb821d61d9fd186f60458ac6ff61" gracePeriod=25
Nov 03 19:50:54.457802 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:54.352329    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"]
Nov 03 19:50:54.457802 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:54.352582    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy" containerID="cri-o://885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa" gracePeriod=30
Nov 03 19:50:54.503165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:54.501129    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:54.537995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:54.537373    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 03 19:50:54.603065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:54.596466    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:54.743063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:54.738231    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:54.765240 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:50:54.765180    5051 logging.go:55] [core] [Channel #1310 SubChannel #1311]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.120:50052", ServerName: "10.132.0.120:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.120:50052: connect: connection refused"
Nov 03 19:50:54.782921 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:50:54.782849    5051 logging.go:55] [core] [Channel #1312 SubChannel #1313]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.119:50052", ServerName: "10.132.0.119:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.119:50052: connect: connection refused"
Nov 03 19:50:54.938094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:54.911262    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:55.180136 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:55.179393    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:50:55.353207 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:55.338172    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Nov 03 19:50:55.360578 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:55.359342    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Nov 03 19:50:55.552066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:55.508154    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:55.707078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:55.706152    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:55.784174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:55.754595    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Nov 03 19:50:55.784174 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:55.775160    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv" podUID="eaab2337-0cb8-4c1b-8155-45ca5ebad934" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.119:50052\" within 1s: context deadline exceeded"
Nov 03 19:50:55.836628 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:55.836582    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh" podUID="f4ebb6dd-147d-4669-b280-2f77e334f18b" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.120:50052\" within 1s: context deadline exceeded"
Nov 03 19:50:55.837032 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:50:55.837012    5051 logging.go:55] [core] [Channel #1310 SubChannel #1311]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.120:50052", ServerName: "10.132.0.120:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.120:50052: operation was canceled"
Nov 03 19:50:55.909094 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:55.893989    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:50:55.994601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:55.987526    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:50:56.233112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.214901    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:50:56.233112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.214978    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:56.337333 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.336388    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:50:56.407525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.367680    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:56.407525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.398511    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:56.660322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.603169    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4" podUID="7ce60e5f-3d38-4e72-b857-bbdb675db1f5" containerName="payload-pre-processing" containerID="cri-o://ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9" gracePeriod=25
Nov 03 19:50:56.679803 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.522496    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:56.693451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.684518    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp" podUID="b6c6d13b-5904-4d07-8534-4b724313bb79" containerName="payload-processing" containerID="cri-o://9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8" gracePeriod=25
Nov 03 19:50:56.809313 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:50:56.774512    5051 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.547s"
Nov 03 19:50:56.809313 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.774679    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:50:56.809313 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.774718    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:50:56.809313 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.774756    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:50:56.809313 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.774789    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:50:56.809313 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.774821    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:50:56.809313 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.774854    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:50:56.809313 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.774888    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:50:56.809313 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.774925    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:50:56.809313 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.777346    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:50:56.999259 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:56.997348    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:57.095075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.018710    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Nov 03 19:50:57.095075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.093229    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:50:57.113323 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.113282    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:50:57.115187 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.115156    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eaab2337-0cb8-4c1b-8155-45ca5ebad934-kube-api-access-qhxxq\" (UniqueName: \"kubernetes.io/projected/eaab2337-0cb8-4c1b-8155-45ca5ebad934-kube-api-access-qhxxq\") pod \"eaab2337-0cb8-4c1b-8155-45ca5ebad934\" (UID: \"eaab2337-0cb8-4c1b-8155-45ca5ebad934\") "
Nov 03 19:50:57.115636 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.115614    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eaab2337-0cb8-4c1b-8155-45ca5ebad934-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaab2337-0cb8-4c1b-8155-45ca5ebad934-plugins-config-volume\") pod \"eaab2337-0cb8-4c1b-8155-45ca5ebad934\" (UID: \"eaab2337-0cb8-4c1b-8155-45ca5ebad934\") "
Nov 03 19:50:57.116913 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.116890    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaab2337-0cb8-4c1b-8155-45ca5ebad934-plugins-config-volume" pod "eaab2337-0cb8-4c1b-8155-45ca5ebad934" (UID: "eaab2337-0cb8-4c1b-8155-45ca5ebad934"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:57.121784 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.120186    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaab2337-0cb8-4c1b-8155-45ca5ebad934-kube-api-access-qhxxq" pod "eaab2337-0cb8-4c1b-8155-45ca5ebad934" (UID: "eaab2337-0cb8-4c1b-8155-45ca5ebad934"). InnerVolumeSpecName "kube-api-access-qhxxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:57.134886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.133281    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:50:57.134886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.133767    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:50:57.178063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.158712    5051 generic.go:356] "Generic (PLEG): container finished" podID="eaab2337-0cb8-4c1b-8155-45ca5ebad934" containerID="82a62337483b2c1c89d7004ff499e635dfcdcb821d61d9fd186f60458ac6ff61" exitCode=0
Nov 03 19:50:57.178063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.158890    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv" event={"ID":"eaab2337-0cb8-4c1b-8155-45ca5ebad934","Type":"ContainerDied","Data":"82a62337483b2c1c89d7004ff499e635dfcdcb821d61d9fd186f60458ac6ff61"}
Nov 03 19:50:57.178063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.158945    5051 scope.go:122] "RemoveContainer" containerID="82a62337483b2c1c89d7004ff499e635dfcdcb821d61d9fd186f60458ac6ff61"
Nov 03 19:50:57.178063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.159142    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"
Nov 03 19:50:57.278066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.276701    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:50:57.290064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.285949    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhxxq\" (UniqueName: \"kubernetes.io/projected/eaab2337-0cb8-4c1b-8155-45ca5ebad934-kube-api-access-qhxxq\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:57.290064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.285985    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaab2337-0cb8-4c1b-8155-45ca5ebad934-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:57.291618 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.291582    5051 generic.go:356] "Generic (PLEG): container finished" podID="f4ebb6dd-147d-4669-b280-2f77e334f18b" containerID="58b19ff9f6f396aa154ed3fa3c6410dc3fc7409fae83ab2a622b122ac6241ba2" exitCode=0
Nov 03 19:50:57.320904 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.320873    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:50:57.410092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.407192    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4ebb6dd-147d-4669-b280-2f77e334f18b-kube-api-access-cl2sv\" (UniqueName: \"kubernetes.io/projected/f4ebb6dd-147d-4669-b280-2f77e334f18b-kube-api-access-cl2sv\") pod \"f4ebb6dd-147d-4669-b280-2f77e334f18b\" (UID: \"f4ebb6dd-147d-4669-b280-2f77e334f18b\") "
Nov 03 19:50:57.410092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.407244    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4ebb6dd-147d-4669-b280-2f77e334f18b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ebb6dd-147d-4669-b280-2f77e334f18b-plugins-config-volume\") pod \"f4ebb6dd-147d-4669-b280-2f77e334f18b\" (UID: \"f4ebb6dd-147d-4669-b280-2f77e334f18b\") "
Nov 03 19:50:57.410092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.408531    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4ebb6dd-147d-4669-b280-2f77e334f18b-plugins-config-volume" pod "f4ebb6dd-147d-4669-b280-2f77e334f18b" (UID: "f4ebb6dd-147d-4669-b280-2f77e334f18b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:57.416989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.416959    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ebb6dd-147d-4669-b280-2f77e334f18b-kube-api-access-cl2sv" pod "f4ebb6dd-147d-4669-b280-2f77e334f18b" (UID: "f4ebb6dd-147d-4669-b280-2f77e334f18b"). InnerVolumeSpecName "kube-api-access-cl2sv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:57.436272 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.436240    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh" event={"ID":"f4ebb6dd-147d-4669-b280-2f77e334f18b","Type":"ContainerDied","Data":"58b19ff9f6f396aa154ed3fa3c6410dc3fc7409fae83ab2a622b122ac6241ba2"}
Nov 03 19:50:57.436400 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.436332    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:50:57.436400 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.436372    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:50:57.436400 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.436387    5051 scope.go:122] "RemoveContainer" containerID="58b19ff9f6f396aa154ed3fa3c6410dc3fc7409fae83ab2a622b122ac6241ba2"
Nov 03 19:50:57.436567 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.436405    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:50:57.437009 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.436992    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:50:57.509943 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.509904    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl2sv\" (UniqueName: \"kubernetes.io/projected/f4ebb6dd-147d-4669-b280-2f77e334f18b-kube-api-access-cl2sv\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:57.510135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.510118    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ebb6dd-147d-4669-b280-2f77e334f18b-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:57.820066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.819969    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"]
Nov 03 19:50:57.823819 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:57.823793    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-477cd8-55cbc48845-msklv"]
Nov 03 19:50:58.011361 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.011324    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"]
Nov 03 19:50:58.011606 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.011583    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t" podUID="92cd8215-515e-43e5-94eb-99ef8add9480" containerName="maas-api" containerID="cri-o://067cc6293c6bd2f897baf1f35faca96c7a38af091f4364281471812d05399ae5" gracePeriod=30
Nov 03 19:50:58.303928 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.303895    5051 generic.go:356] "Generic (PLEG): container finished" podID="92cd8215-515e-43e5-94eb-99ef8add9480" containerID="067cc6293c6bd2f897baf1f35faca96c7a38af091f4364281471812d05399ae5" exitCode=0
Nov 03 19:50:58.304328 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.304031    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t" event={"ID":"92cd8215-515e-43e5-94eb-99ef8add9480","Type":"ContainerDied","Data":"067cc6293c6bd2f897baf1f35faca96c7a38af091f4364281471812d05399ae5"}
Nov 03 19:50:58.313933 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.306305    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"
Nov 03 19:50:58.313933 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.306360    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh" event={"ID":"f4ebb6dd-147d-4669-b280-2f77e334f18b","Type":"ContainerDied","Data":"409f29a5efbce68b0c337c9370d92986defff4c46a2bf79939d8d775dc165a0b"}
Nov 03 19:50:58.331357 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.331335    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"]
Nov 03 19:50:58.373542 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.373492    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-477cd8-64dc98d555-wp4dh"]
Nov 03 19:50:58.488826 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.488794    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:50:58.538643 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.538610    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92cd8215-515e-43e5-94eb-99ef8add9480-kube-api-access-mslkc\" (UniqueName: \"kubernetes.io/projected/92cd8215-515e-43e5-94eb-99ef8add9480-kube-api-access-mslkc\") pod \"92cd8215-515e-43e5-94eb-99ef8add9480\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") "
Nov 03 19:50:58.538909 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.538886    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92cd8215-515e-43e5-94eb-99ef8add9480-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92cd8215-515e-43e5-94eb-99ef8add9480-openshift-service-ca\") pod \"92cd8215-515e-43e5-94eb-99ef8add9480\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") "
Nov 03 19:50:58.539084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.539065    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-metrics-certs\") pod \"92cd8215-515e-43e5-94eb-99ef8add9480\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") "
Nov 03 19:50:58.539215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.539199    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-maas-api-tls\") pod \"92cd8215-515e-43e5-94eb-99ef8add9480\" (UID: \"92cd8215-515e-43e5-94eb-99ef8add9480\") "
Nov 03 19:50:58.621140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.605795    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92cd8215-515e-43e5-94eb-99ef8add9480-openshift-service-ca" pod "92cd8215-515e-43e5-94eb-99ef8add9480" (UID: "92cd8215-515e-43e5-94eb-99ef8add9480"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:58.638714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.638681    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92cd8215-515e-43e5-94eb-99ef8add9480-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:58.678225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.678185    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-metrics-certs" pod "92cd8215-515e-43e5-94eb-99ef8add9480" (UID: "92cd8215-515e-43e5-94eb-99ef8add9480"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:50:58.719261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.716359    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-maas-api-tls" pod "92cd8215-515e-43e5-94eb-99ef8add9480" (UID: "92cd8215-515e-43e5-94eb-99ef8add9480"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:50:58.790066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.785994    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:58.790066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.786035    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92cd8215-515e-43e5-94eb-99ef8add9480-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:58.793394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.793258    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92cd8215-515e-43e5-94eb-99ef8add9480-kube-api-access-mslkc" pod "92cd8215-515e-43e5-94eb-99ef8add9480" (UID: "92cd8215-515e-43e5-94eb-99ef8add9480"). InnerVolumeSpecName "kube-api-access-mslkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:58.887679 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:58.887640    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mslkc\" (UniqueName: \"kubernetes.io/projected/92cd8215-515e-43e5-94eb-99ef8add9480-kube-api-access-mslkc\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:59.232014 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.231972    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eaab2337-0cb8-4c1b-8155-45ca5ebad934" path="/var/lib/kubelet/pods/eaab2337-0cb8-4c1b-8155-45ca5ebad934/volumes"
Nov 03 19:50:59.232667 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.232642    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4ebb6dd-147d-4669-b280-2f77e334f18b" path="/var/lib/kubelet/pods/f4ebb6dd-147d-4669-b280-2f77e334f18b/volumes"
Nov 03 19:50:59.318403 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.318375    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"
Nov 03 19:50:59.318835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.318404    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t" event={"ID":"92cd8215-515e-43e5-94eb-99ef8add9480","Type":"ContainerDied","Data":"edee234d46d0b9fb3b4a2d17962513299417ea82874ff8db920ecfba343b41f0"}
Nov 03 19:50:59.318835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.318477    5051 scope.go:122] "RemoveContainer" containerID="067cc6293c6bd2f897baf1f35faca96c7a38af091f4364281471812d05399ae5"
Nov 03 19:50:59.371075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.366102    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"]
Nov 03 19:50:59.373900 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.373874    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-10e551-554c79698d-2lq2t"]
Nov 03 19:50:59.827480 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.827442    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp" podUID="b6c6d13b-5904-4d07-8534-4b724313bb79" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 19:50:59.828249 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.828212    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4" podUID="7ce60e5f-3d38-4e72-b857-bbdb675db1f5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 19:50:59.866898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.866870    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:50:59.909577 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.909216    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf903bff-eb4a-436e-af4c-9bdae7c21698-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf903bff-eb4a-436e-af4c-9bdae7c21698-istiod-ca-cert\") pod \"bf903bff-eb4a-436e-af4c-9bdae7c21698\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") "
Nov 03 19:50:59.909577 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.909272    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-credential-socket\") pod \"bf903bff-eb4a-436e-af4c-9bdae7c21698\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") "
Nov 03 19:50:59.909577 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.909302    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-socket\") pod \"bf903bff-eb4a-436e-af4c-9bdae7c21698\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") "
Nov 03 19:50:59.909577 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.909389    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-envoy\") pod \"bf903bff-eb4a-436e-af4c-9bdae7c21698\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") "
Nov 03 19:50:59.909577 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.909462    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-kube-api-access-6tq5d\" (UniqueName: \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-kube-api-access-6tq5d\") pod \"bf903bff-eb4a-436e-af4c-9bdae7c21698\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") "
Nov 03 19:50:59.910002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.909600    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-podinfo\") pod \"bf903bff-eb4a-436e-af4c-9bdae7c21698\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") "
Nov 03 19:50:59.910002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.909704    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-certs\") pod \"bf903bff-eb4a-436e-af4c-9bdae7c21698\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") "
Nov 03 19:50:59.910002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.909726    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-socket" pod "bf903bff-eb4a-436e-af4c-9bdae7c21698" (UID: "bf903bff-eb4a-436e-af4c-9bdae7c21698"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:59.910002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.909859    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-token\" (UniqueName: \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-token\") pod \"bf903bff-eb4a-436e-af4c-9bdae7c21698\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") "
Nov 03 19:50:59.910002 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.909920    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-data\") pod \"bf903bff-eb4a-436e-af4c-9bdae7c21698\" (UID: \"bf903bff-eb4a-436e-af4c-9bdae7c21698\") "
Nov 03 19:50:59.910472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.910448    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-certs" pod "bf903bff-eb4a-436e-af4c-9bdae7c21698" (UID: "bf903bff-eb4a-436e-af4c-9bdae7c21698"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:59.910921 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.910900    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-data" pod "bf903bff-eb4a-436e-af4c-9bdae7c21698" (UID: "bf903bff-eb4a-436e-af4c-9bdae7c21698"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:59.911101 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.911068    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-credential-socket" pod "bf903bff-eb4a-436e-af4c-9bdae7c21698" (UID: "bf903bff-eb4a-436e-af4c-9bdae7c21698"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:59.911261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.911200    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:59.911261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.911222    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:59.911261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.911238    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:50:59.911466 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.911349    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf903bff-eb4a-436e-af4c-9bdae7c21698-istiod-ca-cert" pod "bf903bff-eb4a-436e-af4c-9bdae7c21698" (UID: "bf903bff-eb4a-436e-af4c-9bdae7c21698"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:50:59.949316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.949263    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-envoy" pod "bf903bff-eb4a-436e-af4c-9bdae7c21698" (UID: "bf903bff-eb4a-436e-af4c-9bdae7c21698"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:50:59.954219 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.954172    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-podinfo" pod "bf903bff-eb4a-436e-af4c-9bdae7c21698" (UID: "bf903bff-eb4a-436e-af4c-9bdae7c21698"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:50:59.955061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.955019    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-token" pod "bf903bff-eb4a-436e-af4c-9bdae7c21698" (UID: "bf903bff-eb4a-436e-af4c-9bdae7c21698"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:50:59.957954 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:50:59.957920    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-kube-api-access-6tq5d" pod "bf903bff-eb4a-436e-af4c-9bdae7c21698" (UID: "bf903bff-eb4a-436e-af4c-9bdae7c21698"). InnerVolumeSpecName "kube-api-access-6tq5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:00.013539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.013493    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:00.013539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.013535    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tq5d\" (UniqueName: \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-kube-api-access-6tq5d\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:00.013750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.013550    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:00.013750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.013566    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf903bff-eb4a-436e-af4c-9bdae7c21698-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:00.013750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.013580    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf903bff-eb4a-436e-af4c-9bdae7c21698-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:00.013750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.013595    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf903bff-eb4a-436e-af4c-9bdae7c21698-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:00.329408 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.329368    5051 generic.go:356] "Generic (PLEG): container finished" podID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerID="885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa" exitCode=0
Nov 03 19:51:00.329408 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.329416    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"
Nov 03 19:51:00.329964 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.329457    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" event={"ID":"bf903bff-eb4a-436e-af4c-9bdae7c21698","Type":"ContainerDied","Data":"885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa"}
Nov 03 19:51:00.329964 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.329508    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd" event={"ID":"bf903bff-eb4a-436e-af4c-9bdae7c21698","Type":"ContainerDied","Data":"f950147284af93bcd61447746e02c371d4b4096fee52379bc085b621ef041a5a"}
Nov 03 19:51:00.329964 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.329537    5051 scope.go:122] "RemoveContainer" containerID="885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa"
Nov 03 19:51:00.344036 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.344004    5051 scope.go:122] "RemoveContainer" containerID="885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa"
Nov 03 19:51:00.344534 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:00.344459    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa\": container with ID starting with 885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa not found: ID does not exist" containerID="885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa"
Nov 03 19:51:00.344534 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.344502    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa"} err="failed to get container status \"885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa\": rpc error: code = NotFound desc = could not find container \"885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa\": container with ID starting with 885b6ef45586a5572f5fa76685d3da414576d7ebb47128aef8ddade39d1657fa not found: ID does not exist"
Nov 03 19:51:00.358807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.358775    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"]
Nov 03 19:51:00.365886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.365850    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c5f6cb-openshift-default-774487478c-2ndmd"]
Nov 03 19:51:00.665085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.664965    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main" containerID="cri-o://cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9" gracePeriod=45
Nov 03 19:51:00.668816 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:00.668769    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main" containerID="cri-o://483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a" gracePeriod=45
Nov 03 19:51:01.061025 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.061000    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:51:01.064493 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.064436    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:51:01.139172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.139139    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-dshm\") pod \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") "
Nov 03 19:51:01.140141 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.140117    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-dshm\") pod \"0b5a03fb-7018-4e65-984a-719d0284ae34\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") "
Nov 03 19:51:01.140685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.140664    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b5a03fb-7018-4e65-984a-719d0284ae34-kube-api-access-xw95v\" (UniqueName: \"kubernetes.io/projected/0b5a03fb-7018-4e65-984a-719d0284ae34-kube-api-access-xw95v\") pod \"0b5a03fb-7018-4e65-984a-719d0284ae34\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") "
Nov 03 19:51:01.141204 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.141186    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-model-cache\") pod \"0b5a03fb-7018-4e65-984a-719d0284ae34\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") "
Nov 03 19:51:01.141395 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.141380    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-home\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-home\") pod \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") "
Nov 03 19:51:01.141499 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.141485    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-model-cache\") pod \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") "
Nov 03 19:51:01.141665 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.141651    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tmp-dir\") pod \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") "
Nov 03 19:51:01.141787 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.141774    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b5a03fb-7018-4e65-984a-719d0284ae34-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b5a03fb-7018-4e65-984a-719d0284ae34-tls-certs\") pod \"0b5a03fb-7018-4e65-984a-719d0284ae34\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") "
Nov 03 19:51:01.141888 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.141876    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-home\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-home\") pod \"0b5a03fb-7018-4e65-984a-719d0284ae34\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") "
Nov 03 19:51:01.142028 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.142015    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-tmp-dir\") pod \"0b5a03fb-7018-4e65-984a-719d0284ae34\" (UID: \"0b5a03fb-7018-4e65-984a-719d0284ae34\") "
Nov 03 19:51:01.142337 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.142319    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-kube-api-access-vqz5j\" (UniqueName: \"kubernetes.io/projected/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-kube-api-access-vqz5j\") pod \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") "
Nov 03 19:51:01.142444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.142430    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tls-certs\") pod \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\" (UID: \"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11\") "
Nov 03 19:51:01.143855 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.143833    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tmp-dir" pod "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" (UID: "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:01.144121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.144007    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-dshm" pod "0b5a03fb-7018-4e65-984a-719d0284ae34" (UID: "0b5a03fb-7018-4e65-984a-719d0284ae34"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:01.144362 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.144343    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-home" pod "0b5a03fb-7018-4e65-984a-719d0284ae34" (UID: "0b5a03fb-7018-4e65-984a-719d0284ae34"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:01.145730 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.145701    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-model-cache" pod "0b5a03fb-7018-4e65-984a-719d0284ae34" (UID: "0b5a03fb-7018-4e65-984a-719d0284ae34"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:01.145939 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.145919    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-home" pod "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" (UID: "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:01.146155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.146134    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-model-cache" pod "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" (UID: "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:01.146343 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.146322    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-tmp-dir" pod "0b5a03fb-7018-4e65-984a-719d0284ae34" (UID: "0b5a03fb-7018-4e65-984a-719d0284ae34"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:01.147233 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.147202    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b5a03fb-7018-4e65-984a-719d0284ae34-tls-certs" pod "0b5a03fb-7018-4e65-984a-719d0284ae34" (UID: "0b5a03fb-7018-4e65-984a-719d0284ae34"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:01.150858 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.150828    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-kube-api-access-vqz5j" pod "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" (UID: "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11"). InnerVolumeSpecName "kube-api-access-vqz5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:01.157022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.156992    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b5a03fb-7018-4e65-984a-719d0284ae34-kube-api-access-xw95v" pod "0b5a03fb-7018-4e65-984a-719d0284ae34" (UID: "0b5a03fb-7018-4e65-984a-719d0284ae34"). InnerVolumeSpecName "kube-api-access-xw95v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:01.158082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.158057    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-dshm" pod "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" (UID: "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:01.160306 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.160278    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tls-certs" pod "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" (UID: "7ef91b29-e8d8-4003-9a0b-3f5f68d07a11"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:01.233460 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.233425    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92cd8215-515e-43e5-94eb-99ef8add9480" path="/var/lib/kubelet/pods/92cd8215-515e-43e5-94eb-99ef8add9480/volumes"
Nov 03 19:51:01.234008 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.233993    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" path="/var/lib/kubelet/pods/bf903bff-eb4a-436e-af4c-9bdae7c21698/volumes"
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.244900    5051 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-dshm\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.244925    5051 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-dshm\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.244941    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw95v\" (UniqueName: \"kubernetes.io/projected/0b5a03fb-7018-4e65-984a-719d0284ae34-kube-api-access-xw95v\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.244957    5051 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-model-cache\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.244971    5051 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-home\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.244983    5051 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-model-cache\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.244997    5051 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tmp-dir\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.245011    5051 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b5a03fb-7018-4e65-984a-719d0284ae34-tls-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.245022    5051 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-home\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.245076    5051 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b5a03fb-7018-4e65-984a-719d0284ae34-tmp-dir\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.245091    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqz5j\" (UniqueName: \"kubernetes.io/projected/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-kube-api-access-vqz5j\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.245310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.245104    5051 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11-tls-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:01.343075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.342984    5051 generic.go:356] "Generic (PLEG): container finished" podID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerID="cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9" exitCode=0
Nov 03 19:51:01.343545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.343291    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" event={"ID":"0b5a03fb-7018-4e65-984a-719d0284ae34","Type":"ContainerDied","Data":"cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9"}
Nov 03 19:51:01.343545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.343330    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn" event={"ID":"0b5a03fb-7018-4e65-984a-719d0284ae34","Type":"ContainerDied","Data":"3678c026bbc70afc5c2b8867cb0d91178a60476138efc076c2a646a4014089fc"}
Nov 03 19:51:01.343545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.343352    5051 scope.go:122] "RemoveContainer" containerID="cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9"
Nov 03 19:51:01.343545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.343490    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"
Nov 03 19:51:01.346873 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.346847    5051 generic.go:356] "Generic (PLEG): container finished" podID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerID="483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a" exitCode=0
Nov 03 19:51:01.347016 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.346917    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" event={"ID":"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11","Type":"ContainerDied","Data":"483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a"}
Nov 03 19:51:01.347016 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.346944    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn" event={"ID":"7ef91b29-e8d8-4003-9a0b-3f5f68d07a11","Type":"ContainerDied","Data":"13ddeb7ea07a9a124ba1d30afdfc3b1e854cf1679dc09d7dddec70fb809ec3e6"}
Nov 03 19:51:01.347534 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.347508    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"
Nov 03 19:51:01.360290 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.360266    5051 scope.go:122] "RemoveContainer" containerID="cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9"
Nov 03 19:51:01.360684 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:01.360659    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9\": container with ID starting with cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9 not found: ID does not exist" containerID="cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9"
Nov 03 19:51:01.360750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.360696    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9"} err="failed to get container status \"cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9\": rpc error: code = NotFound desc = could not find container \"cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9\": container with ID starting with cf923bf3157d0f7ee0a5a10c2cd2197f9650c51315cb05b837110be24886f2e9 not found: ID does not exist"
Nov 03 19:51:01.360750 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.360734    5051 scope.go:122] "RemoveContainer" containerID="483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a"
Nov 03 19:51:01.372621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.372593    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"]
Nov 03 19:51:01.373172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.373149    5051 scope.go:122] "RemoveContainer" containerID="483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a"
Nov 03 19:51:01.373467 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:01.373446    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a\": container with ID starting with 483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a not found: ID does not exist" containerID="483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a"
Nov 03 19:51:01.373534 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.373475    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a"} err="failed to get container status \"483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a\": rpc error: code = NotFound desc = could not find container \"483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a\": container with ID starting with 483c6fe15a22399aff7350fa766c0915aee7de312d26979689bc30c91338637a not found: ID does not exist"
Nov 03 19:51:01.380524 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.380499    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/distinct-w0-ef8f84-kserve-94c58d9d4-zx9bn"]
Nov 03 19:51:01.393421 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.393393    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"]
Nov 03 19:51:01.403839 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:01.403808    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-ef8f84/premium-w0-ef8f84-kserve-7b7b76f977-djsqn"]
Nov 03 19:51:02.769371 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:02.769335    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:51:03.231621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:03.231593    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" path="/var/lib/kubelet/pods/0b5a03fb-7018-4e65-984a-719d0284ae34/volumes"
Nov 03 19:51:03.232108 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:03.232091    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" path="/var/lib/kubelet/pods/7ef91b29-e8d8-4003-9a0b-3f5f68d07a11/volumes"
Nov 03 19:51:03.374072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:03.374007    5051 generic.go:356] "Generic (PLEG): container finished" podID="8eade3a5-b467-4599-b909-cb3306407caa" containerID="7337569bc2f97c82612abf258988701a6ad700826c67d1c0007ebe7bfecb4bd8" exitCode=0
Nov 03 19:51:03.374245 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:03.374105    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" event={"ID":"8eade3a5-b467-4599-b909-cb3306407caa","Type":"ContainerDied","Data":"7337569bc2f97c82612abf258988701a6ad700826c67d1c0007ebe7bfecb4bd8"}
Nov 03 19:51:03.374687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:03.374660    5051 scope.go:122] "RemoveContainer" containerID="7337569bc2f97c82612abf258988701a6ad700826c67d1c0007ebe7bfecb4bd8"
Nov 03 19:51:03.376592 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:03.376561    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:03.376695 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:03.376632    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:03.376695 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:03.376667    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:04.243714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.243687    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"]
Nov 03 19:51:04.244233 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.243915    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9" podUID="30d163b8-cbff-40ac-85a7-5f8438d82d4a" containerName="maas-api" containerID="cri-o://b7ddcee88e52d50cfc1cef5b562ffa7993198435a8a4814f0ddf22f723631baa" gracePeriod=30
Nov 03 19:51:04.388008 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.387962    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" event={"ID":"8eade3a5-b467-4599-b909-cb3306407caa","Type":"ContainerStarted","Data":"f06310fbc6fcc232bfa2b5feb47bcaa2ffa4b2b1c955fbc4b8b22cc44beaf599"}
Nov 03 19:51:04.396740 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.396708    5051 generic.go:356] "Generic (PLEG): container finished" podID="d7655d6f-386a-4364-9343-6e56877d3faa" containerID="219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2" exitCode=0
Nov 03 19:51:04.396905 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.396872    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" event={"ID":"d7655d6f-386a-4364-9343-6e56877d3faa","Type":"ContainerDied","Data":"219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2"}
Nov 03 19:51:04.397331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.397306    5051 scope.go:122] "RemoveContainer" containerID="219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2"
Nov 03 19:51:04.399125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.399094    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:04.399250 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.399173    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:04.399250 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.399209    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:04.399383 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.399316    5051 generic.go:356] "Generic (PLEG): container finished" podID="30d163b8-cbff-40ac-85a7-5f8438d82d4a" containerID="b7ddcee88e52d50cfc1cef5b562ffa7993198435a8a4814f0ddf22f723631baa" exitCode=0
Nov 03 19:51:04.399451 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.399433    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9" event={"ID":"30d163b8-cbff-40ac-85a7-5f8438d82d4a","Type":"ContainerDied","Data":"b7ddcee88e52d50cfc1cef5b562ffa7993198435a8a4814f0ddf22f723631baa"}
Nov 03 19:51:04.618233 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.617918    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:51:04.726472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.724889    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-metrics-certs\") pod \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") "
Nov 03 19:51:04.726472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.725015    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30d163b8-cbff-40ac-85a7-5f8438d82d4a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30d163b8-cbff-40ac-85a7-5f8438d82d4a-openshift-service-ca\") pod \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") "
Nov 03 19:51:04.726472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.725101    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-maas-api-tls\") pod \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") "
Nov 03 19:51:04.726472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.725193    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30d163b8-cbff-40ac-85a7-5f8438d82d4a-kube-api-access-5rlhr\" (UniqueName: \"kubernetes.io/projected/30d163b8-cbff-40ac-85a7-5f8438d82d4a-kube-api-access-5rlhr\") pod \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\" (UID: \"30d163b8-cbff-40ac-85a7-5f8438d82d4a\") "
Nov 03 19:51:04.813903 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.807556    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30d163b8-cbff-40ac-85a7-5f8438d82d4a-openshift-service-ca" pod "30d163b8-cbff-40ac-85a7-5f8438d82d4a" (UID: "30d163b8-cbff-40ac-85a7-5f8438d82d4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:04.874744 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.871312    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30d163b8-cbff-40ac-85a7-5f8438d82d4a-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:04.901385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.901336    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-maas-api-tls" pod "30d163b8-cbff-40ac-85a7-5f8438d82d4a" (UID: "30d163b8-cbff-40ac-85a7-5f8438d82d4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:04.901778 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.901757    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30d163b8-cbff-40ac-85a7-5f8438d82d4a-kube-api-access-5rlhr" pod "30d163b8-cbff-40ac-85a7-5f8438d82d4a" (UID: "30d163b8-cbff-40ac-85a7-5f8438d82d4a"). InnerVolumeSpecName "kube-api-access-5rlhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:04.901964 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.901947    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-metrics-certs" pod "30d163b8-cbff-40ac-85a7-5f8438d82d4a" (UID: "30d163b8-cbff-40ac-85a7-5f8438d82d4a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:04.976363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.976326    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:04.976363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.976360    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30d163b8-cbff-40ac-85a7-5f8438d82d4a-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:04.976363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:04.976372    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rlhr\" (UniqueName: \"kubernetes.io/projected/30d163b8-cbff-40ac-85a7-5f8438d82d4a-kube-api-access-5rlhr\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:05.187126 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.187032    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:51:05.188087 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.188055    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 03 19:51:05.188216 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.188115    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 03 19:51:05.408619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.408584    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" event={"ID":"d7655d6f-386a-4364-9343-6e56877d3faa","Type":"ContainerStarted","Data":"01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd"}
Nov 03 19:51:05.410008 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.409988    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"
Nov 03 19:51:05.410161 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.410029    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9" event={"ID":"30d163b8-cbff-40ac-85a7-5f8438d82d4a","Type":"ContainerDied","Data":"b0445619d97b38ea859a2a14ccbbb4780155272ddb7f2a34440df670dca45e24"}
Nov 03 19:51:05.410161 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.410110    5051 scope.go:122] "RemoveContainer" containerID="b7ddcee88e52d50cfc1cef5b562ffa7993198435a8a4814f0ddf22f723631baa"
Nov 03 19:51:05.470989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.470959    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"]
Nov 03 19:51:05.474457 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.474435    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-477cd8-846c5dc698-hnxh9"]
Nov 03 19:51:05.865254 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.865135    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:51:05.865530 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.865495    5051 patch_prober.go:29] interesting pod/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 03 19:51:05.865624 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:05.865578    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 03 19:51:06.192661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:06.192553    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 03 19:51:06.192839 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:06.192651    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 03 19:51:06.864062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:06.862748    5051 patch_prober.go:29] interesting pod/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 03 19:51:06.864062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:06.862816    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 03 19:51:06.895074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:06.894410    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"]
Nov 03 19:51:06.895074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:06.895005    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy" containerID="cri-o://80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543" gracePeriod=30
Nov 03 19:51:07.187975 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:07.187883    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 03 19:51:07.188267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:07.188236    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 03 19:51:07.232661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:07.232631    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30d163b8-cbff-40ac-85a7-5f8438d82d4a" path="/var/lib/kubelet/pods/30d163b8-cbff-40ac-85a7-5f8438d82d4a/volumes"
Nov 03 19:51:07.861065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:07.859897    5051 patch_prober.go:29] interesting pod/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 03 19:51:07.861065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:07.859957    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 03 19:51:07.994795 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:07.994756    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"]
Nov 03 19:51:08.006162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:08.006135    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"]
Nov 03 19:51:08.090642 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:08.090610    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:51:08.215938 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:08.215900    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 03 19:51:08.216145 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:08.215977    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 03 19:51:08.865065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:08.864233    5051 patch_prober.go:29] interesting pod/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 03 19:51:08.865065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:08.864310    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 03 19:51:09.190496 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.190412    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 03 19:51:09.197980 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.197945    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 03 19:51:09.482167 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.482130    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"]
Nov 03 19:51:09.504092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.483484    5051 cpu_manager.go:414] "containerMap: removing container" podUID="eaab2337-0cb8-4c1b-8155-45ca5ebad934" containerName="payload-processing"
Nov 03 19:51:09.504315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.504293    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eaab2337-0cb8-4c1b-8155-45ca5ebad934" containerName="payload-processing"
Nov 03 19:51:09.504415 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.504403    5051 cpu_manager.go:414] "containerMap: removing container" podUID="f4ebb6dd-147d-4669-b280-2f77e334f18b" containerName="payload-pre-processing"
Nov 03 19:51:09.504489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.504478    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4ebb6dd-147d-4669-b280-2f77e334f18b" containerName="payload-pre-processing"
Nov 03 19:51:09.504595 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.504586    5051 cpu_manager.go:414] "containerMap: removing container" podUID="92cd8215-515e-43e5-94eb-99ef8add9480" containerName="maas-api"
Nov 03 19:51:09.504664 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.504656    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92cd8215-515e-43e5-94eb-99ef8add9480" containerName="maas-api"
Nov 03 19:51:09.504746 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.504738    5051 cpu_manager.go:414] "containerMap: removing container" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy"
Nov 03 19:51:09.504816 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.504806    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy"
Nov 03 19:51:09.504887 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.504878    5051 cpu_manager.go:414] "containerMap: removing container" podUID="30d163b8-cbff-40ac-85a7-5f8438d82d4a" containerName="maas-api"
Nov 03 19:51:09.504959 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.504950    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30d163b8-cbff-40ac-85a7-5f8438d82d4a" containerName="maas-api"
Nov 03 19:51:09.505065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.505055    5051 cpu_manager.go:414] "containerMap: removing container" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main"
Nov 03 19:51:09.505147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.505138    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main"
Nov 03 19:51:09.505238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.505229    5051 cpu_manager.go:414] "containerMap: removing container" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main"
Nov 03 19:51:09.505306 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.505298    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main"
Nov 03 19:51:09.505598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.505584    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf903bff-eb4a-436e-af4c-9bdae7c21698" containerName="istio-proxy"
Nov 03 19:51:09.505682 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.505674    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="30d163b8-cbff-40ac-85a7-5f8438d82d4a" containerName="maas-api"
Nov 03 19:51:09.505759 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.505751    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b5a03fb-7018-4e65-984a-719d0284ae34" containerName="main"
Nov 03 19:51:09.505832 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.505824    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ef91b29-e8d8-4003-9a0b-3f5f68d07a11" containerName="main"
Nov 03 19:51:09.505909 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.505901    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="eaab2337-0cb8-4c1b-8155-45ca5ebad934" containerName="payload-processing"
Nov 03 19:51:09.505981 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.505973    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4ebb6dd-147d-4669-b280-2f77e334f18b" containerName="payload-pre-processing"
Nov 03 19:51:09.506080 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.506070    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="92cd8215-515e-43e5-94eb-99ef8add9480" containerName="maas-api"
Nov 03 19:51:09.517069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.515359    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.517673 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.517643    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"]
Nov 03 19:51:09.520212 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.520186    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-c6a08e-serving-cert\""
Nov 03 19:51:09.770122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.766069    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-metrics-certs\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.770122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.766248    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz7dq\" (UniqueName: \"kubernetes.io/projected/b740565b-db74-4f04-b916-0b62e4d14ea0-kube-api-access-nz7dq\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.770122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.766344    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b740565b-db74-4f04-b916-0b62e4d14ea0-openshift-service-ca\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.770122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.766399    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-maas-api-tls\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.826298 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.826254    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4" podUID="7ce60e5f-3d38-4e72-b857-bbdb675db1f5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 19:51:09.826641 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.826259    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp" podUID="b6c6d13b-5904-4d07-8534-4b724313bb79" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 19:51:09.861062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.860441    5051 patch_prober.go:29] interesting pod/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 03 19:51:09.861062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.860494    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 03 19:51:09.870829 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.870786    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz7dq\" (UniqueName: \"kubernetes.io/projected/b740565b-db74-4f04-b916-0b62e4d14ea0-kube-api-access-nz7dq\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.891472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.871419    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b740565b-db74-4f04-b916-0b62e4d14ea0-openshift-service-ca\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.891930 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.891909    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-maas-api-tls\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.892482 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.892462    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b740565b-db74-4f04-b916-0b62e4d14ea0-openshift-service-ca\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.892658 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.892639    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-metrics-certs\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.899409 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.899378    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-metrics-certs\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.900628 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.900297    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz7dq\" (UniqueName: \"kubernetes.io/projected/b740565b-db74-4f04-b916-0b62e4d14ea0-kube-api-access-nz7dq\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:09.900628 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:09.900600    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-maas-api-tls\") pod \"maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:10.136106 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:10.134307    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:10.189089 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:10.188441    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body=
Nov 03 19:51:10.189089 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:10.188517    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused"
Nov 03 19:51:10.610327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:10.610292    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:51:10.705067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:10.703511    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"]
Nov 03 19:51:10.726528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:10.726496    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q" event={"ID":"b740565b-db74-4f04-b916-0b62e4d14ea0","Type":"ContainerStarted","Data":"3ff77012ea8c8f8a5d2149828d4555b2c54e79d08e14211125ace0c73f55b4a9"}
Nov 03 19:51:10.872801 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:10.872709    5051 patch_prober.go:29] interesting pod/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Nov 03 19:51:10.873097 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:10.873029    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Nov 03 19:51:11.193297 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:11.193272    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:51:11.194598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:11.194561    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:51:11.736571 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:11.736534    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q" event={"ID":"b740565b-db74-4f04-b916-0b62e4d14ea0","Type":"ContainerStarted","Data":"d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07"}
Nov 03 19:51:11.737012 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:11.736636    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:11.758845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:11.758805    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q" podStartSLOduration=2.274618073 podStartE2EDuration="2.758792617s" totalImagesPullingTime="484.174544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:51:11.757031127 +0000 UTC m=+1472.665053076" watchObservedRunningTime="2026-11-03 19:51:11.758792617 +0000 UTC m=+1472.666814555"
Nov 03 19:51:11.864242 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:11.864217    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:51:11.865344 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:11.865312    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:51:12.210503 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.210479    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:51:12.367318 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.367224    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-socket\") pod \"262d3eee-5594-4d75-83e2-1eb72166f710\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") "
Nov 03 19:51:12.367318 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.367305    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/262d3eee-5594-4d75-83e2-1eb72166f710-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262d3eee-5594-4d75-83e2-1eb72166f710-istio-podinfo\") pod \"262d3eee-5594-4d75-83e2-1eb72166f710\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") "
Nov 03 19:51:12.367574 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.367404    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-credential-socket\") pod \"262d3eee-5594-4d75-83e2-1eb72166f710\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") "
Nov 03 19:51:12.367574 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.367447    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-certs\") pod \"262d3eee-5594-4d75-83e2-1eb72166f710\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") "
Nov 03 19:51:12.367574 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.367477    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-kube-api-access-ls9tg\" (UniqueName: \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-kube-api-access-ls9tg\") pod \"262d3eee-5594-4d75-83e2-1eb72166f710\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") "
Nov 03 19:51:12.367574 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.367543    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-istio-token\" (UniqueName: \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-istio-token\") pod \"262d3eee-5594-4d75-83e2-1eb72166f710\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") "
Nov 03 19:51:12.367794 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.367636    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-socket" pod "262d3eee-5594-4d75-83e2-1eb72166f710" (UID: "262d3eee-5594-4d75-83e2-1eb72166f710"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:12.367794 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.367725    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-credential-socket" pod "262d3eee-5594-4d75-83e2-1eb72166f710" (UID: "262d3eee-5594-4d75-83e2-1eb72166f710"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:12.367794 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.367743    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/262d3eee-5594-4d75-83e2-1eb72166f710-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262d3eee-5594-4d75-83e2-1eb72166f710-istiod-ca-cert\") pod \"262d3eee-5594-4d75-83e2-1eb72166f710\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") "
Nov 03 19:51:12.368000 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.367910    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-envoy\") pod \"262d3eee-5594-4d75-83e2-1eb72166f710\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") "
Nov 03 19:51:12.368070 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.367993    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-certs" pod "262d3eee-5594-4d75-83e2-1eb72166f710" (UID: "262d3eee-5594-4d75-83e2-1eb72166f710"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:12.368122 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.368070    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-data\") pod \"262d3eee-5594-4d75-83e2-1eb72166f710\" (UID: \"262d3eee-5594-4d75-83e2-1eb72166f710\") "
Nov 03 19:51:12.368372 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.368348    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-data" pod "262d3eee-5594-4d75-83e2-1eb72166f710" (UID: "262d3eee-5594-4d75-83e2-1eb72166f710"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:12.368432 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.368377    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/262d3eee-5594-4d75-83e2-1eb72166f710-istiod-ca-cert" pod "262d3eee-5594-4d75-83e2-1eb72166f710" (UID: "262d3eee-5594-4d75-83e2-1eb72166f710"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:12.368959 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.368931    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262d3eee-5594-4d75-83e2-1eb72166f710-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:12.369096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.368962    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:12.369096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.368977    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:12.369096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.368992    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:12.369096 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.369006    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:12.369752 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.369728    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/262d3eee-5594-4d75-83e2-1eb72166f710-istio-podinfo" pod "262d3eee-5594-4d75-83e2-1eb72166f710" (UID: "262d3eee-5594-4d75-83e2-1eb72166f710"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:51:12.370228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.370207    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-kube-api-access-ls9tg" pod "262d3eee-5594-4d75-83e2-1eb72166f710" (UID: "262d3eee-5594-4d75-83e2-1eb72166f710"). InnerVolumeSpecName "kube-api-access-ls9tg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:12.370327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.370297    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-envoy" pod "262d3eee-5594-4d75-83e2-1eb72166f710" (UID: "262d3eee-5594-4d75-83e2-1eb72166f710"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:12.370385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.370330    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-istio-token" pod "262d3eee-5594-4d75-83e2-1eb72166f710" (UID: "262d3eee-5594-4d75-83e2-1eb72166f710"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:12.470721 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.470687    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262d3eee-5594-4d75-83e2-1eb72166f710-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:12.470721 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.470716    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls9tg\" (UniqueName: \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-kube-api-access-ls9tg\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:12.470721 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.470726    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262d3eee-5594-4d75-83e2-1eb72166f710-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:12.470944 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.470735    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262d3eee-5594-4d75-83e2-1eb72166f710-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:12.758129 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.757537    5051 generic.go:356] "Generic (PLEG): container finished" podID="262d3eee-5594-4d75-83e2-1eb72166f710" containerID="80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543" exitCode=0
Nov 03 19:51:12.762065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.758891    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"
Nov 03 19:51:12.764692 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.764126    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" event={"ID":"262d3eee-5594-4d75-83e2-1eb72166f710","Type":"ContainerDied","Data":"80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543"}
Nov 03 19:51:12.764692 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.764188    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw" event={"ID":"262d3eee-5594-4d75-83e2-1eb72166f710","Type":"ContainerDied","Data":"7848d3d5e75b1e8771e69092601b982fae3204a9371eb517244f5bd075d4822a"}
Nov 03 19:51:12.764692 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.764220    5051 scope.go:122] "RemoveContainer" containerID="80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543"
Nov 03 19:51:12.780927 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.780906    5051 scope.go:122] "RemoveContainer" containerID="eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6"
Nov 03 19:51:12.792540 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.792513    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"]
Nov 03 19:51:12.793832 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.793814    5051 scope.go:122] "RemoveContainer" containerID="80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543"
Nov 03 19:51:12.794197 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:12.794171    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543\": container with ID starting with 80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543 not found: ID does not exist" containerID="80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543"
Nov 03 19:51:12.794291 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.794209    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543"} err="failed to get container status \"80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543\": rpc error: code = NotFound desc = could not find container \"80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543\": container with ID starting with 80d877d7bff0d681bf1de6052ca5a4e100069ab71f40895a62cd7715a950c543 not found: ID does not exist"
Nov 03 19:51:12.794291 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.794237    5051 scope.go:122] "RemoveContainer" containerID="eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6"
Nov 03 19:51:12.794524 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:12.794500    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6\": container with ID starting with eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6 not found: ID does not exist" containerID="eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6"
Nov 03 19:51:12.794579 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.794536    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6"} err="failed to get container status \"eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6\": rpc error: code = NotFound desc = could not find container \"eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6\": container with ID starting with eb07a78bdee7ca0cdafe8dfe713a3c0d2c8082fd04f50dacff915e23ca54d7c6 not found: ID does not exist"
Nov 03 19:51:12.797142 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.797120    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-10e551-openshift-default-978c6c8f5-h48fw"]
Nov 03 19:51:12.995546 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:12.995505    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr" podUID="ff0336ae-47dd-45db-9a74-1aaf1bbcb47d" containerName="payload-processing" containerID="cri-o://b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97" gracePeriod=25
Nov 03 19:51:13.009116 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.006490    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm" podUID="9a1a4117-0b43-4d57-8023-25bed714d499" containerName="payload-pre-processing" containerID="cri-o://478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675" gracePeriod=25
Nov 03 19:51:13.246150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.245092    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" path="/var/lib/kubelet/pods/262d3eee-5594-4d75-83e2-1eb72166f710/volumes"
Nov 03 19:51:13.294749 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.294461    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"]
Nov 03 19:51:13.294749 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.294704    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy" containerID="cri-o://7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06" gracePeriod=30
Nov 03 19:51:13.302649 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.302621    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"]
Nov 03 19:51:13.302875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.302830    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj" podUID="965226d2-5ae1-4bdf-97b3-6726e9c035d7" containerName="istio-proxy" containerID="cri-o://9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7" gracePeriod=30
Nov 03 19:51:13.342422 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.342393    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:51:13.384331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.384305    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:51:13.515217 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.514981    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a1a4117-0b43-4d57-8023-25bed714d499-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a1a4117-0b43-4d57-8023-25bed714d499-plugins-config-volume\") pod \"9a1a4117-0b43-4d57-8023-25bed714d499\" (UID: \"9a1a4117-0b43-4d57-8023-25bed714d499\") "
Nov 03 19:51:13.515217 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.515174    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-plugins-config-volume\") pod \"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d\" (UID: \"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d\") "
Nov 03 19:51:13.515497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.515231    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-kube-api-access-ljkgh\" (UniqueName: \"kubernetes.io/projected/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-kube-api-access-ljkgh\") pod \"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d\" (UID: \"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d\") "
Nov 03 19:51:13.515497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.515370    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a1a4117-0b43-4d57-8023-25bed714d499-kube-api-access-798hg\" (UniqueName: \"kubernetes.io/projected/9a1a4117-0b43-4d57-8023-25bed714d499-kube-api-access-798hg\") pod \"9a1a4117-0b43-4d57-8023-25bed714d499\" (UID: \"9a1a4117-0b43-4d57-8023-25bed714d499\") "
Nov 03 19:51:13.516802 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.516773    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-plugins-config-volume" pod "ff0336ae-47dd-45db-9a74-1aaf1bbcb47d" (UID: "ff0336ae-47dd-45db-9a74-1aaf1bbcb47d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:13.517354 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.517335    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a1a4117-0b43-4d57-8023-25bed714d499-plugins-config-volume" pod "9a1a4117-0b43-4d57-8023-25bed714d499" (UID: "9a1a4117-0b43-4d57-8023-25bed714d499"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:13.596332 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.596240    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a1a4117-0b43-4d57-8023-25bed714d499-kube-api-access-798hg" pod "9a1a4117-0b43-4d57-8023-25bed714d499" (UID: "9a1a4117-0b43-4d57-8023-25bed714d499"). InnerVolumeSpecName "kube-api-access-798hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:13.596682 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.596656    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-kube-api-access-ljkgh" pod "ff0336ae-47dd-45db-9a74-1aaf1bbcb47d" (UID: "ff0336ae-47dd-45db-9a74-1aaf1bbcb47d"). InnerVolumeSpecName "kube-api-access-ljkgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:13.617198 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.617133    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:13.617198 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.617160    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljkgh\" (UniqueName: \"kubernetes.io/projected/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d-kube-api-access-ljkgh\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:13.617198 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.617170    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-798hg\" (UniqueName: \"kubernetes.io/projected/9a1a4117-0b43-4d57-8023-25bed714d499-kube-api-access-798hg\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:13.617198 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.617178    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a1a4117-0b43-4d57-8023-25bed714d499-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:13.766620 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.766585    5051 generic.go:356] "Generic (PLEG): container finished" podID="9a1a4117-0b43-4d57-8023-25bed714d499" containerID="478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675" exitCode=0
Nov 03 19:51:13.767067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.766645    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"
Nov 03 19:51:13.767067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.766663    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm" event={"ID":"9a1a4117-0b43-4d57-8023-25bed714d499","Type":"ContainerDied","Data":"478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675"}
Nov 03 19:51:13.767067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.766739    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm" event={"ID":"9a1a4117-0b43-4d57-8023-25bed714d499","Type":"ContainerDied","Data":"5a091c4bf8eced8dead2e1010bf4ece53d7386b52b4b551af70da3005c35f93a"}
Nov 03 19:51:13.767067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.766762    5051 scope.go:122] "RemoveContainer" containerID="478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675"
Nov 03 19:51:13.769410 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.769383    5051 generic.go:356] "Generic (PLEG): container finished" podID="ff0336ae-47dd-45db-9a74-1aaf1bbcb47d" containerID="b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97" exitCode=0
Nov 03 19:51:13.769532 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.769431    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"
Nov 03 19:51:13.769532 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.769425    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr" event={"ID":"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d","Type":"ContainerDied","Data":"b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97"}
Nov 03 19:51:13.769624 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.769529    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr" event={"ID":"ff0336ae-47dd-45db-9a74-1aaf1bbcb47d","Type":"ContainerDied","Data":"d842035b88d2013f07988af9b8f8fc4aec93cb7d98325efae64b83e543381b5f"}
Nov 03 19:51:13.778192 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.777982    5051 scope.go:122] "RemoveContainer" containerID="478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675"
Nov 03 19:51:13.778283 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:13.778263    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675\": container with ID starting with 478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675 not found: ID does not exist" containerID="478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675"
Nov 03 19:51:13.778322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.778292    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675"} err="failed to get container status \"478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675\": rpc error: code = NotFound desc = could not find container \"478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675\": container with ID starting with 478048fed02573df57362b63ab208dae784256b91050a6545c820f65c6403675 not found: ID does not exist"
Nov 03 19:51:13.778322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.778309    5051 scope.go:122] "RemoveContainer" containerID="b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97"
Nov 03 19:51:13.788191 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.788169    5051 scope.go:122] "RemoveContainer" containerID="b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97"
Nov 03 19:51:13.788442 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:13.788422    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97\": container with ID starting with b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97 not found: ID does not exist" containerID="b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97"
Nov 03 19:51:13.788499 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.788449    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97"} err="failed to get container status \"b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97\": rpc error: code = NotFound desc = could not find container \"b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97\": container with ID starting with b1e5e2fbbbc7d8313a5b1c7f7446c8837acffbc71fa53cc8722faa22d83dde97 not found: ID does not exist"
Nov 03 19:51:13.794440 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.794420    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"]
Nov 03 19:51:13.800880 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.800858    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-300831-568489dd59-fnjgm"]
Nov 03 19:51:13.812867 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.812845    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"]
Nov 03 19:51:13.815974 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:13.815957    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-300831-7d6f76fff7-shvqr"]
Nov 03 19:51:15.230316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:15.230280    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a1a4117-0b43-4d57-8023-25bed714d499" path="/var/lib/kubelet/pods/9a1a4117-0b43-4d57-8023-25bed714d499/volumes"
Nov 03 19:51:15.230779 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:15.230764    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff0336ae-47dd-45db-9a74-1aaf1bbcb47d" path="/var/lib/kubelet/pods/ff0336ae-47dd-45db-9a74-1aaf1bbcb47d/volumes"
Nov 03 19:51:17.173365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.173305    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:51:17.275246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.275216    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-plugins-config-volume\") pod \"7ce60e5f-3d38-4e72-b857-bbdb675db1f5\" (UID: \"7ce60e5f-3d38-4e72-b857-bbdb675db1f5\") "
Nov 03 19:51:17.275474 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.275454    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-kube-api-access-htlpm\" (UniqueName: \"kubernetes.io/projected/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-kube-api-access-htlpm\") pod \"7ce60e5f-3d38-4e72-b857-bbdb675db1f5\" (UID: \"7ce60e5f-3d38-4e72-b857-bbdb675db1f5\") "
Nov 03 19:51:17.275707 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.275681    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-plugins-config-volume" pod "7ce60e5f-3d38-4e72-b857-bbdb675db1f5" (UID: "7ce60e5f-3d38-4e72-b857-bbdb675db1f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:17.276323 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.276298    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:17.281222 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.281194    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-kube-api-access-htlpm" pod "7ce60e5f-3d38-4e72-b857-bbdb675db1f5" (UID: "7ce60e5f-3d38-4e72-b857-bbdb675db1f5"). InnerVolumeSpecName "kube-api-access-htlpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:17.322473 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.322453    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:51:17.377835 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.377795    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6c6d13b-5904-4d07-8534-4b724313bb79-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6c6d13b-5904-4d07-8534-4b724313bb79-plugins-config-volume\") pod \"b6c6d13b-5904-4d07-8534-4b724313bb79\" (UID: \"b6c6d13b-5904-4d07-8534-4b724313bb79\") "
Nov 03 19:51:17.378027 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.377893    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6c6d13b-5904-4d07-8534-4b724313bb79-kube-api-access-kst6f\" (UniqueName: \"kubernetes.io/projected/b6c6d13b-5904-4d07-8534-4b724313bb79-kube-api-access-kst6f\") pod \"b6c6d13b-5904-4d07-8534-4b724313bb79\" (UID: \"b6c6d13b-5904-4d07-8534-4b724313bb79\") "
Nov 03 19:51:17.378995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.378962    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htlpm\" (UniqueName: \"kubernetes.io/projected/7ce60e5f-3d38-4e72-b857-bbdb675db1f5-kube-api-access-htlpm\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:17.380112 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.379818    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6c6d13b-5904-4d07-8534-4b724313bb79-plugins-config-volume" pod "b6c6d13b-5904-4d07-8534-4b724313bb79" (UID: "b6c6d13b-5904-4d07-8534-4b724313bb79"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:17.391283 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.391249    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6c6d13b-5904-4d07-8534-4b724313bb79-kube-api-access-kst6f" pod "b6c6d13b-5904-4d07-8534-4b724313bb79" (UID: "b6c6d13b-5904-4d07-8534-4b724313bb79"). InnerVolumeSpecName "kube-api-access-kst6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:17.480661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.480629    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6c6d13b-5904-4d07-8534-4b724313bb79-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:17.480661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.480657    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kst6f\" (UniqueName: \"kubernetes.io/projected/b6c6d13b-5904-4d07-8534-4b724313bb79-kube-api-access-kst6f\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:17.765726 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.765643    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:51:17.824364 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.824331    5051 generic.go:356] "Generic (PLEG): container finished" podID="7ce60e5f-3d38-4e72-b857-bbdb675db1f5" containerID="ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9" exitCode=0
Nov 03 19:51:17.824589 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.824416    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"
Nov 03 19:51:17.824684 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.824584    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4" event={"ID":"7ce60e5f-3d38-4e72-b857-bbdb675db1f5","Type":"ContainerDied","Data":"ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9"}
Nov 03 19:51:17.824684 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.824637    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4" event={"ID":"7ce60e5f-3d38-4e72-b857-bbdb675db1f5","Type":"ContainerDied","Data":"419ab7c5ce32c3d9f98bb4e7fc3901aba47e43d7b334383ce92ff2b1163764d4"}
Nov 03 19:51:17.824684 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.824668    5051 scope.go:122] "RemoveContainer" containerID="ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9"
Nov 03 19:51:17.838217 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.838187    5051 generic.go:356] "Generic (PLEG): container finished" podID="b6c6d13b-5904-4d07-8534-4b724313bb79" containerID="9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8" exitCode=0
Nov 03 19:51:17.838331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.838280    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp" event={"ID":"b6c6d13b-5904-4d07-8534-4b724313bb79","Type":"ContainerDied","Data":"9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8"}
Nov 03 19:51:17.838331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.838325    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp" event={"ID":"b6c6d13b-5904-4d07-8534-4b724313bb79","Type":"ContainerDied","Data":"ca711630074bd39f972b19b000df94ec3c31c4669a79d47a71988ee2570bf29d"}
Nov 03 19:51:17.838454 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.838393    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"
Nov 03 19:51:17.846495 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.846475    5051 scope.go:122] "RemoveContainer" containerID="ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9"
Nov 03 19:51:17.846910 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:17.846758    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9\": container with ID starting with ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9 not found: ID does not exist" containerID="ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9"
Nov 03 19:51:17.846910 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.846797    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9"} err="failed to get container status \"ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9\": rpc error: code = NotFound desc = could not find container \"ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9\": container with ID starting with ef2f355c7de549c2e01e5cd5548c5d67de6b34f587f7e26f879c7ef55c4bd0b9 not found: ID does not exist"
Nov 03 19:51:17.846910 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.846818    5051 scope.go:122] "RemoveContainer" containerID="9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8"
Nov 03 19:51:17.861331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.861308    5051 scope.go:122] "RemoveContainer" containerID="9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8"
Nov 03 19:51:17.861661 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:17.861641    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8\": container with ID starting with 9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8 not found: ID does not exist" containerID="9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8"
Nov 03 19:51:17.861765 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.861665    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8"} err="failed to get container status \"9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8\": rpc error: code = NotFound desc = could not find container \"9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8\": container with ID starting with 9514a701e9fa5aafbe466e64d81d1d41d678c6c18ed64bd9d2a2314db4b090f8 not found: ID does not exist"
Nov 03 19:51:17.873630 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.873604    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"]
Nov 03 19:51:17.877293 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.877272    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ef8f84-fdc996767-sqxz4"]
Nov 03 19:51:17.887554 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.887535    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"]
Nov 03 19:51:17.891405 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:17.891383    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ef8f84-78d795f49-z2zwp"]
Nov 03 19:51:18.624807 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.624777    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:51:18.703728 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.703703    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:51:18.710331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.710304    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-kube-api-access-6cvjc\" (UniqueName: \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-kube-api-access-6cvjc\") pod \"082b8718-d755-4a33-9cc5-f1a157b846ec\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") "
Nov 03 19:51:18.710475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.710360    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-data\") pod \"082b8718-d755-4a33-9cc5-f1a157b846ec\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") "
Nov 03 19:51:18.710475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.710408    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/082b8718-d755-4a33-9cc5-f1a157b846ec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082b8718-d755-4a33-9cc5-f1a157b846ec-istiod-ca-cert\") pod \"082b8718-d755-4a33-9cc5-f1a157b846ec\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") "
Nov 03 19:51:18.710610 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.710533    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-envoy\") pod \"082b8718-d755-4a33-9cc5-f1a157b846ec\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") "
Nov 03 19:51:18.710676 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.710616    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-token\" (UniqueName: \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-token\") pod \"082b8718-d755-4a33-9cc5-f1a157b846ec\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") "
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.710672    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-podinfo\") pod \"082b8718-d755-4a33-9cc5-f1a157b846ec\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") "
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.710693    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-data" pod "082b8718-d755-4a33-9cc5-f1a157b846ec" (UID: "082b8718-d755-4a33-9cc5-f1a157b846ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.710713    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-socket\") pod \"082b8718-d755-4a33-9cc5-f1a157b846ec\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") "
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.710811    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-credential-socket\") pod \"082b8718-d755-4a33-9cc5-f1a157b846ec\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") "
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.710841    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-certs\") pod \"082b8718-d755-4a33-9cc5-f1a157b846ec\" (UID: \"082b8718-d755-4a33-9cc5-f1a157b846ec\") "
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.711439    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082b8718-d755-4a33-9cc5-f1a157b846ec-istiod-ca-cert" pod "082b8718-d755-4a33-9cc5-f1a157b846ec" (UID: "082b8718-d755-4a33-9cc5-f1a157b846ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.711493    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-credential-socket" pod "082b8718-d755-4a33-9cc5-f1a157b846ec" (UID: "082b8718-d755-4a33-9cc5-f1a157b846ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.711673    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-socket" pod "082b8718-d755-4a33-9cc5-f1a157b846ec" (UID: "082b8718-d755-4a33-9cc5-f1a157b846ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.711795    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.711815    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082b8718-d755-4a33-9cc5-f1a157b846ec-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.711832    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.711973 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.711846    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.712641 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.712101    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-certs" pod "082b8718-d755-4a33-9cc5-f1a157b846ec" (UID: "082b8718-d755-4a33-9cc5-f1a157b846ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:18.714507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.714479    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-podinfo" pod "082b8718-d755-4a33-9cc5-f1a157b846ec" (UID: "082b8718-d755-4a33-9cc5-f1a157b846ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:51:18.714666 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.714537    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-token" pod "082b8718-d755-4a33-9cc5-f1a157b846ec" (UID: "082b8718-d755-4a33-9cc5-f1a157b846ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:18.714666 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.714550    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-envoy" pod "082b8718-d755-4a33-9cc5-f1a157b846ec" (UID: "082b8718-d755-4a33-9cc5-f1a157b846ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:18.714666 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.714575    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-kube-api-access-6cvjc" pod "082b8718-d755-4a33-9cc5-f1a157b846ec" (UID: "082b8718-d755-4a33-9cc5-f1a157b846ec"). InnerVolumeSpecName "kube-api-access-6cvjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:18.813166 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813135    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-data\") pod \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") "
Nov 03 19:51:18.813338 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813265    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-credential-socket\") pod \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") "
Nov 03 19:51:18.813338 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813287    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-socket\") pod \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") "
Nov 03 19:51:18.813338 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813324    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istiod-ca-cert\") pod \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") "
Nov 03 19:51:18.813446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813360    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-certs\") pod \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") "
Nov 03 19:51:18.813446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813394    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-token\" (UniqueName: \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-token\") pod \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") "
Nov 03 19:51:18.813536 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813430    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-data" pod "965226d2-5ae1-4bdf-97b3-6726e9c035d7" (UID: "965226d2-5ae1-4bdf-97b3-6726e9c035d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:18.813536 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813476    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-envoy\") pod \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") "
Nov 03 19:51:18.813536 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813508    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-kube-api-access-4bpwg\" (UniqueName: \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-kube-api-access-4bpwg\") pod \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") "
Nov 03 19:51:18.813536 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813511    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-credential-socket" pod "965226d2-5ae1-4bdf-97b3-6726e9c035d7" (UID: "965226d2-5ae1-4bdf-97b3-6726e9c035d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:18.813717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813596    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-podinfo\") pod \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\" (UID: \"965226d2-5ae1-4bdf-97b3-6726e9c035d7\") "
Nov 03 19:51:18.813717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813658    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-socket" pod "965226d2-5ae1-4bdf-97b3-6726e9c035d7" (UID: "965226d2-5ae1-4bdf-97b3-6726e9c035d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:18.813827 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813747    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-certs" pod "965226d2-5ae1-4bdf-97b3-6726e9c035d7" (UID: "965226d2-5ae1-4bdf-97b3-6726e9c035d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:18.813827 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.813765    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istiod-ca-cert" pod "965226d2-5ae1-4bdf-97b3-6726e9c035d7" (UID: "965226d2-5ae1-4bdf-97b3-6726e9c035d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:18.814178 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.814156    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.814178 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.814176    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.814178 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.814185    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.814449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.814197    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cvjc\" (UniqueName: \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-kube-api-access-6cvjc\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.814449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.814206    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.814449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.814220    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.814449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.814238    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.814449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.814255    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082b8718-d755-4a33-9cc5-f1a157b846ec-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.814449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.814267    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.814449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.814279    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.815676 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.815648    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-podinfo" pod "965226d2-5ae1-4bdf-97b3-6726e9c035d7" (UID: "965226d2-5ae1-4bdf-97b3-6726e9c035d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:51:18.815785 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.815771    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-token" pod "965226d2-5ae1-4bdf-97b3-6726e9c035d7" (UID: "965226d2-5ae1-4bdf-97b3-6726e9c035d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:18.815822 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.815793    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-envoy" pod "965226d2-5ae1-4bdf-97b3-6726e9c035d7" (UID: "965226d2-5ae1-4bdf-97b3-6726e9c035d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:18.815862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.815840    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-kube-api-access-4bpwg" pod "965226d2-5ae1-4bdf-97b3-6726e9c035d7" (UID: "965226d2-5ae1-4bdf-97b3-6726e9c035d7"). InnerVolumeSpecName "kube-api-access-4bpwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:18.870857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.870762    5051 generic.go:356] "Generic (PLEG): container finished" podID="965226d2-5ae1-4bdf-97b3-6726e9c035d7" containerID="9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7" exitCode=0
Nov 03 19:51:18.870857 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.870807    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"
Nov 03 19:51:18.871095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.870854    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj" event={"ID":"965226d2-5ae1-4bdf-97b3-6726e9c035d7","Type":"ContainerDied","Data":"9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7"}
Nov 03 19:51:18.871095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.870967    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj" event={"ID":"965226d2-5ae1-4bdf-97b3-6726e9c035d7","Type":"ContainerDied","Data":"ee4a3282a95e56499b7d54eb3b07810b8b8080ec90d1e763be04be1c28447f84"}
Nov 03 19:51:18.871095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.870992    5051 scope.go:122] "RemoveContainer" containerID="9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7"
Nov 03 19:51:18.874156 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.874133    5051 generic.go:356] "Generic (PLEG): container finished" podID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerID="7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06" exitCode=0
Nov 03 19:51:18.874277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.874196    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"
Nov 03 19:51:18.874277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.874228    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" event={"ID":"082b8718-d755-4a33-9cc5-f1a157b846ec","Type":"ContainerDied","Data":"7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06"}
Nov 03 19:51:18.874390 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.874279    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz" event={"ID":"082b8718-d755-4a33-9cc5-f1a157b846ec","Type":"ContainerDied","Data":"a1e32ef01a7d5879dc52dd5c4b00c2a1483555641eeaa75ca384b70ab21755ae"}
Nov 03 19:51:18.884551 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.884525    5051 scope.go:122] "RemoveContainer" containerID="9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7"
Nov 03 19:51:18.885178 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:18.885155    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7\": container with ID starting with 9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7 not found: ID does not exist" containerID="9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7"
Nov 03 19:51:18.885273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.885185    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7"} err="failed to get container status \"9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7\": rpc error: code = NotFound desc = could not find container \"9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7\": container with ID starting with 9f69f14ed5235500669dd5f286d2ac96e3ebf49b9e85ef27df637a7c72414bc7 not found: ID does not exist"
Nov 03 19:51:18.885273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.885200    5051 scope.go:122] "RemoveContainer" containerID="7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06"
Nov 03 19:51:18.895048 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.895024    5051 scope.go:122] "RemoveContainer" containerID="eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841"
Nov 03 19:51:18.904446 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.904424    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"]
Nov 03 19:51:18.905461 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.905446    5051 scope.go:122] "RemoveContainer" containerID="7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06"
Nov 03 19:51:18.906206 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:18.906184    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06\": container with ID starting with 7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06 not found: ID does not exist" containerID="7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06"
Nov 03 19:51:18.906298 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.906212    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06"} err="failed to get container status \"7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06\": rpc error: code = NotFound desc = could not find container \"7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06\": container with ID starting with 7d9197471f68b554d9151e13382b15d4633b3833c7af8889cccfa0072e7d8f06 not found: ID does not exist"
Nov 03 19:51:18.906298 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.906230    5051 scope.go:122] "RemoveContainer" containerID="eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841"
Nov 03 19:51:18.906596 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:18.906565    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841\": container with ID starting with eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841 not found: ID does not exist" containerID="eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841"
Nov 03 19:51:18.906667 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.906606    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841"} err="failed to get container status \"eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841\": rpc error: code = NotFound desc = could not find container \"eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841\": container with ID starting with eaf0e5af76efa1921064f2deabcb091b61ecaf0bc3abe5de2fa2dd824b738841 not found: ID does not exist"
Nov 03 19:51:18.909965 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.909946    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-z9qhz"]
Nov 03 19:51:18.916079 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.916056    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.916164 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.916083    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.916164 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.916100    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bpwg\" (UniqueName: \"kubernetes.io/projected/965226d2-5ae1-4bdf-97b3-6726e9c035d7-kube-api-access-4bpwg\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.916164 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.916114    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965226d2-5ae1-4bdf-97b3-6726e9c035d7-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:18.927185 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.927164    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"]
Nov 03 19:51:18.933214 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:18.933194    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-477cd8-openshift-default-7b4f875647-pfcnj"]
Nov 03 19:51:19.229806 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:19.229777    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" path="/var/lib/kubelet/pods/082b8718-d755-4a33-9cc5-f1a157b846ec/volumes"
Nov 03 19:51:19.230234 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:19.230218    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ce60e5f-3d38-4e72-b857-bbdb675db1f5" path="/var/lib/kubelet/pods/7ce60e5f-3d38-4e72-b857-bbdb675db1f5/volumes"
Nov 03 19:51:19.230675 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:19.230662    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="965226d2-5ae1-4bdf-97b3-6726e9c035d7" path="/var/lib/kubelet/pods/965226d2-5ae1-4bdf-97b3-6726e9c035d7/volumes"
Nov 03 19:51:19.231363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:19.231348    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6c6d13b-5904-4d07-8534-4b724313bb79" path="/var/lib/kubelet/pods/b6c6d13b-5904-4d07-8534-4b724313bb79/volumes"
Nov 03 19:51:27.253604 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.253572    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"]
Nov 03 19:51:27.254162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.253772    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk" podUID="3843274d-f1ec-4da5-806f-e841042b92f6" containerName="maas-api" containerID="cri-o://441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e" gracePeriod=30
Nov 03 19:51:27.681507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.681480    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:51:27.821473 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.816465    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3843274d-f1ec-4da5-806f-e841042b92f6-kube-api-access-hwf8t\" (UniqueName: \"kubernetes.io/projected/3843274d-f1ec-4da5-806f-e841042b92f6-kube-api-access-hwf8t\") pod \"3843274d-f1ec-4da5-806f-e841042b92f6\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") "
Nov 03 19:51:27.821473 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.816532    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-maas-api-tls\") pod \"3843274d-f1ec-4da5-806f-e841042b92f6\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") "
Nov 03 19:51:27.821473 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.816699    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-metrics-certs\") pod \"3843274d-f1ec-4da5-806f-e841042b92f6\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") "
Nov 03 19:51:27.821473 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.816975    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3843274d-f1ec-4da5-806f-e841042b92f6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3843274d-f1ec-4da5-806f-e841042b92f6-openshift-service-ca\") pod \"3843274d-f1ec-4da5-806f-e841042b92f6\" (UID: \"3843274d-f1ec-4da5-806f-e841042b92f6\") "
Nov 03 19:51:27.821473 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.821125    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3843274d-f1ec-4da5-806f-e841042b92f6-openshift-service-ca" pod "3843274d-f1ec-4da5-806f-e841042b92f6" (UID: "3843274d-f1ec-4da5-806f-e841042b92f6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:27.833625 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.826473    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3843274d-f1ec-4da5-806f-e841042b92f6-kube-api-access-hwf8t" pod "3843274d-f1ec-4da5-806f-e841042b92f6" (UID: "3843274d-f1ec-4da5-806f-e841042b92f6"). InnerVolumeSpecName "kube-api-access-hwf8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:27.844320 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.844271    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-metrics-certs" pod "3843274d-f1ec-4da5-806f-e841042b92f6" (UID: "3843274d-f1ec-4da5-806f-e841042b92f6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:27.849841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.849812    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-maas-api-tls" pod "3843274d-f1ec-4da5-806f-e841042b92f6" (UID: "3843274d-f1ec-4da5-806f-e841042b92f6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:27.922265 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.922236    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3843274d-f1ec-4da5-806f-e841042b92f6-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:27.922265 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.922265    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwf8t\" (UniqueName: \"kubernetes.io/projected/3843274d-f1ec-4da5-806f-e841042b92f6-kube-api-access-hwf8t\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:27.922415 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.922275    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:27.922415 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.922285    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3843274d-f1ec-4da5-806f-e841042b92f6-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:27.954996 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.954950    5051 generic.go:356] "Generic (PLEG): container finished" podID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerID="d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4" exitCode=0
Nov 03 19:51:27.955202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.955180    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" event={"ID":"4b0a399e-8e57-4d85-bdb5-94577f83a19c","Type":"ContainerDied","Data":"d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4"}
Nov 03 19:51:27.955687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.955667    5051 scope.go:122] "RemoveContainer" containerID="d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4"
Nov 03 19:51:27.957508 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.957481    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:27.957628 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.957558    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:27.957628 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.957594    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:27.958219 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.958032    5051 generic.go:356] "Generic (PLEG): container finished" podID="3843274d-f1ec-4da5-806f-e841042b92f6" containerID="441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e" exitCode=0
Nov 03 19:51:27.958219 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.958113    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk" event={"ID":"3843274d-f1ec-4da5-806f-e841042b92f6","Type":"ContainerDied","Data":"441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e"}
Nov 03 19:51:27.958219 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.958127    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"
Nov 03 19:51:27.958219 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.958160    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk" event={"ID":"3843274d-f1ec-4da5-806f-e841042b92f6","Type":"ContainerDied","Data":"9b65edaceb15a39a26d231c727c41139dc92f9ad6e54cd00a68f77bdb0188343"}
Nov 03 19:51:27.958219 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.958189    5051 scope.go:122] "RemoveContainer" containerID="441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e"
Nov 03 19:51:27.973034 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.973013    5051 scope.go:122] "RemoveContainer" containerID="441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e"
Nov 03 19:51:27.973330 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:27.973311    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e\": container with ID starting with 441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e not found: ID does not exist" containerID="441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e"
Nov 03 19:51:27.973394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:27.973338    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e"} err="failed to get container status \"441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e\": rpc error: code = NotFound desc = could not find container \"441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e\": container with ID starting with 441795987bfe8f1f54944cd03cbd7613b311fd00f48fc69dbd27c41e7f77026e not found: ID does not exist"
Nov 03 19:51:28.006740 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:28.002686    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"]
Nov 03 19:51:28.008113 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:28.008086    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-5444b0-58f999dddc-7h2wk"]
Nov 03 19:51:28.973227 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:28.973100    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" event={"ID":"4b0a399e-8e57-4d85-bdb5-94577f83a19c","Type":"ContainerStarted","Data":"14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3"}
Nov 03 19:51:29.229954 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:29.229863    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3843274d-f1ec-4da5-806f-e841042b92f6" path="/var/lib/kubelet/pods/3843274d-f1ec-4da5-806f-e841042b92f6/volumes"
Nov 03 19:51:29.691715 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:29.691620    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:51:29.691870 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:29.691846    5051 patch_prober.go:29] interesting pod/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 03 19:51:29.691928 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:29.691902    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 03 19:51:30.596545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:30.596514    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"]
Nov 03 19:51:30.597145 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:30.597118    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy" containerID="cri-o://5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef" gracePeriod=30
Nov 03 19:51:30.691938 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:30.691898    5051 patch_prober.go:29] interesting pod/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 03 19:51:30.692110 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:30.691961    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 03 19:51:31.692414 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:31.692375    5051 patch_prober.go:29] interesting pod/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 03 19:51:31.692890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:31.692439    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 03 19:51:32.691991 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:32.691957    5051 patch_prober.go:29] interesting pod/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 03 19:51:32.692196 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:32.692011    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 03 19:51:32.814889 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:32.814822    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:51:33.279910 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:33.277963    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"]
Nov 03 19:51:33.279910 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:33.278262    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq" podUID="abcde2a0-cda0-4635-89f3-b791dee9537e" containerName="maas-api" containerID="cri-o://9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92" gracePeriod=30
Nov 03 19:51:33.700062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:33.699798    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"]
Nov 03 19:51:33.700254 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:33.700088    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv" podUID="526cdbab-900d-4990-9a52-deb0b1277d9a" containerName="maas-api" containerID="cri-o://ad4993ae185a01e8cff8b3e668cc57100bc09e1dacee2dd3d205dc92fcb38cd7" gracePeriod=30
Nov 03 19:51:33.774120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:33.738899    5051 patch_prober.go:29] interesting pod/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 03 19:51:33.774120 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:33.738979    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 03 19:51:33.929033 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:33.929009    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:51:34.028887 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.028850    5051 generic.go:356] "Generic (PLEG): container finished" podID="526cdbab-900d-4990-9a52-deb0b1277d9a" containerID="ad4993ae185a01e8cff8b3e668cc57100bc09e1dacee2dd3d205dc92fcb38cd7" exitCode=0
Nov 03 19:51:34.029246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.029176    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv" event={"ID":"526cdbab-900d-4990-9a52-deb0b1277d9a","Type":"ContainerDied","Data":"ad4993ae185a01e8cff8b3e668cc57100bc09e1dacee2dd3d205dc92fcb38cd7"}
Nov 03 19:51:34.031175 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.031152    5051 generic.go:356] "Generic (PLEG): container finished" podID="abcde2a0-cda0-4635-89f3-b791dee9537e" containerID="9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92" exitCode=0
Nov 03 19:51:34.031345 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.031190    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq" event={"ID":"abcde2a0-cda0-4635-89f3-b791dee9537e","Type":"ContainerDied","Data":"9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92"}
Nov 03 19:51:34.031447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.031381    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq" event={"ID":"abcde2a0-cda0-4635-89f3-b791dee9537e","Type":"ContainerDied","Data":"ac5b4bd9a3ed068b21c4089a76c0d1ccf8d35ae296c6f806bcded1a26e048ccf"}
Nov 03 19:51:34.031447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.031415    5051 scope.go:122] "RemoveContainer" containerID="9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92"
Nov 03 19:51:34.031447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.031217    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"
Nov 03 19:51:34.045066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.043989    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/abcde2a0-cda0-4635-89f3-b791dee9537e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abcde2a0-cda0-4635-89f3-b791dee9537e-openshift-service-ca\") pod \"abcde2a0-cda0-4635-89f3-b791dee9537e\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") "
Nov 03 19:51:34.045066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.044102    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-maas-api-tls\") pod \"abcde2a0-cda0-4635-89f3-b791dee9537e\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") "
Nov 03 19:51:34.045066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.044320    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-metrics-certs\") pod \"abcde2a0-cda0-4635-89f3-b791dee9537e\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") "
Nov 03 19:51:34.045066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.044348    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abcde2a0-cda0-4635-89f3-b791dee9537e-kube-api-access-4sfkc\" (UniqueName: \"kubernetes.io/projected/abcde2a0-cda0-4635-89f3-b791dee9537e-kube-api-access-4sfkc\") pod \"abcde2a0-cda0-4635-89f3-b791dee9537e\" (UID: \"abcde2a0-cda0-4635-89f3-b791dee9537e\") "
Nov 03 19:51:34.100074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.099976    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abcde2a0-cda0-4635-89f3-b791dee9537e-openshift-service-ca" pod "abcde2a0-cda0-4635-89f3-b791dee9537e" (UID: "abcde2a0-cda0-4635-89f3-b791dee9537e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:34.197070 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.180771    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abcde2a0-cda0-4635-89f3-b791dee9537e-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:34.199143 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.198934    5051 scope.go:122] "RemoveContainer" containerID="9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92"
Nov 03 19:51:34.199618 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:34.199590    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92\": container with ID starting with 9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92 not found: ID does not exist" containerID="9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92"
Nov 03 19:51:34.199722 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.199632    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92"} err="failed to get container status \"9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92\": rpc error: code = NotFound desc = could not find container \"9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92\": container with ID starting with 9ae6bf2095c5bf5087d22304f223a0d37f2aec70d197dd26879c488d2fa80a92 not found: ID does not exist"
Nov 03 19:51:34.221316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.221258    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-metrics-certs" pod "abcde2a0-cda0-4635-89f3-b791dee9537e" (UID: "abcde2a0-cda0-4635-89f3-b791dee9537e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:34.227198 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.224306    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-maas-api-tls" pod "abcde2a0-cda0-4635-89f3-b791dee9537e" (UID: "abcde2a0-cda0-4635-89f3-b791dee9537e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:34.227198 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.224469    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abcde2a0-cda0-4635-89f3-b791dee9537e-kube-api-access-4sfkc" pod "abcde2a0-cda0-4635-89f3-b791dee9537e" (UID: "abcde2a0-cda0-4635-89f3-b791dee9537e"). InnerVolumeSpecName "kube-api-access-4sfkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:34.302315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.302282    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:34.302470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.302317    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sfkc\" (UniqueName: \"kubernetes.io/projected/abcde2a0-cda0-4635-89f3-b791dee9537e-kube-api-access-4sfkc\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:34.302470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.302337    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abcde2a0-cda0-4635-89f3-b791dee9537e-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:34.372954 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.372927    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:51:34.387789 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.387746    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"]
Nov 03 19:51:34.391245 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.391218    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-300831-7b7674495d-nx5dq"]
Nov 03 19:51:34.505095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.505036    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/526cdbab-900d-4990-9a52-deb0b1277d9a-kube-api-access-mphgg\" (UniqueName: \"kubernetes.io/projected/526cdbab-900d-4990-9a52-deb0b1277d9a-kube-api-access-mphgg\") pod \"526cdbab-900d-4990-9a52-deb0b1277d9a\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") "
Nov 03 19:51:34.505271 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.505243    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-metrics-certs\") pod \"526cdbab-900d-4990-9a52-deb0b1277d9a\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") "
Nov 03 19:51:34.505470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.505448    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/526cdbab-900d-4990-9a52-deb0b1277d9a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/526cdbab-900d-4990-9a52-deb0b1277d9a-openshift-service-ca\") pod \"526cdbab-900d-4990-9a52-deb0b1277d9a\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") "
Nov 03 19:51:34.505587 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.505550    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-maas-api-tls\") pod \"526cdbab-900d-4990-9a52-deb0b1277d9a\" (UID: \"526cdbab-900d-4990-9a52-deb0b1277d9a\") "
Nov 03 19:51:34.505841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.505808    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/526cdbab-900d-4990-9a52-deb0b1277d9a-openshift-service-ca" pod "526cdbab-900d-4990-9a52-deb0b1277d9a" (UID: "526cdbab-900d-4990-9a52-deb0b1277d9a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:34.506470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.506445    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/526cdbab-900d-4990-9a52-deb0b1277d9a-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:34.507475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.507439    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-metrics-certs" pod "526cdbab-900d-4990-9a52-deb0b1277d9a" (UID: "526cdbab-900d-4990-9a52-deb0b1277d9a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:34.507666 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.507648    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/526cdbab-900d-4990-9a52-deb0b1277d9a-kube-api-access-mphgg" pod "526cdbab-900d-4990-9a52-deb0b1277d9a" (UID: "526cdbab-900d-4990-9a52-deb0b1277d9a"). InnerVolumeSpecName "kube-api-access-mphgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:34.508155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.508133    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-maas-api-tls" pod "526cdbab-900d-4990-9a52-deb0b1277d9a" (UID: "526cdbab-900d-4990-9a52-deb0b1277d9a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:34.608403 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.608363    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:34.608403 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.608394    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mphgg\" (UniqueName: \"kubernetes.io/projected/526cdbab-900d-4990-9a52-deb0b1277d9a-kube-api-access-mphgg\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:34.608403 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.608408    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/526cdbab-900d-4990-9a52-deb0b1277d9a-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:34.876720 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.876692    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:51:34.877095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.877072    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:51:34.878621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:34.878597    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:51:35.042207 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:35.042178    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"
Nov 03 19:51:35.042638 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:35.042209    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv" event={"ID":"526cdbab-900d-4990-9a52-deb0b1277d9a","Type":"ContainerDied","Data":"154c36169f28c8803f99437e333877afed3a3dc76556fbd2afdb1dcdd8e2b70b"}
Nov 03 19:51:35.042638 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:35.042264    5051 scope.go:122] "RemoveContainer" containerID="ad4993ae185a01e8cff8b3e668cc57100bc09e1dacee2dd3d205dc92fcb38cd7"
Nov 03 19:51:35.085538 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:35.085511    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"]
Nov 03 19:51:35.091081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:35.091055    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8cff38-b5545f876-bjpjv"]
Nov 03 19:51:35.230895 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:35.230860    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="526cdbab-900d-4990-9a52-deb0b1277d9a" path="/var/lib/kubelet/pods/526cdbab-900d-4990-9a52-deb0b1277d9a/volumes"
Nov 03 19:51:35.231417 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:35.231402    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abcde2a0-cda0-4635-89f3-b791dee9537e" path="/var/lib/kubelet/pods/abcde2a0-cda0-4635-89f3-b791dee9537e/volumes"
Nov 03 19:51:35.891877 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:35.891851    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:51:36.028254 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.028221    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-podinfo\") pod \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") "
Nov 03 19:51:36.028418 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.028267    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istiod-ca-cert\") pod \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") "
Nov 03 19:51:36.028418 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.028310    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-token\" (UniqueName: \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-token\") pod \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") "
Nov 03 19:51:36.028418 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.028373    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-socket\") pod \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") "
Nov 03 19:51:36.028418 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.028409    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-envoy\") pod \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") "
Nov 03 19:51:36.028672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.028452    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-credential-socket\") pod \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") "
Nov 03 19:51:36.028672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.028571    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-kube-api-access-qz8qv\" (UniqueName: \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-kube-api-access-qz8qv\") pod \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") "
Nov 03 19:51:36.028672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.028601    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-certs\") pod \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") "
Nov 03 19:51:36.028672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.028664    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-data\") pod \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\" (UID: \"e2ae4144-2795-4fee-b8c7-9a5859c55d35\") "
Nov 03 19:51:36.028879 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.028762    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-socket" pod "e2ae4144-2795-4fee-b8c7-9a5859c55d35" (UID: "e2ae4144-2795-4fee-b8c7-9a5859c55d35"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:36.028879 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.028768    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istiod-ca-cert" pod "e2ae4144-2795-4fee-b8c7-9a5859c55d35" (UID: "e2ae4144-2795-4fee-b8c7-9a5859c55d35"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:36.029077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.029023    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-certs" pod "e2ae4144-2795-4fee-b8c7-9a5859c55d35" (UID: "e2ae4144-2795-4fee-b8c7-9a5859c55d35"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:36.029178 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.029074    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-credential-socket" pod "e2ae4144-2795-4fee-b8c7-9a5859c55d35" (UID: "e2ae4144-2795-4fee-b8c7-9a5859c55d35"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:36.029230 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.029198    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-data" pod "e2ae4144-2795-4fee-b8c7-9a5859c55d35" (UID: "e2ae4144-2795-4fee-b8c7-9a5859c55d35"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:36.029496 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.029477    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:36.029586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.029497    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:36.029586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.029507    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:36.029586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.029518    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:36.029586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.029531    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:36.031013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.030982    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-podinfo" pod "e2ae4144-2795-4fee-b8c7-9a5859c55d35" (UID: "e2ae4144-2795-4fee-b8c7-9a5859c55d35"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:51:36.031144 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.031009    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-token" pod "e2ae4144-2795-4fee-b8c7-9a5859c55d35" (UID: "e2ae4144-2795-4fee-b8c7-9a5859c55d35"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:36.031144 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.031089    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-kube-api-access-qz8qv" pod "e2ae4144-2795-4fee-b8c7-9a5859c55d35" (UID: "e2ae4144-2795-4fee-b8c7-9a5859c55d35"). InnerVolumeSpecName "kube-api-access-qz8qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:36.031317 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.031293    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-envoy" pod "e2ae4144-2795-4fee-b8c7-9a5859c55d35" (UID: "e2ae4144-2795-4fee-b8c7-9a5859c55d35"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:36.058060 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.053435    5051 generic.go:356] "Generic (PLEG): container finished" podID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerID="5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef" exitCode=0
Nov 03 19:51:36.058060 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.053596    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" event={"ID":"e2ae4144-2795-4fee-b8c7-9a5859c55d35","Type":"ContainerDied","Data":"5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef"}
Nov 03 19:51:36.058060 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.053688    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb" event={"ID":"e2ae4144-2795-4fee-b8c7-9a5859c55d35","Type":"ContainerDied","Data":"bf394c1cb253bb83af959e898f939c09978ed52b1c3233ba538a186932c3b26e"}
Nov 03 19:51:36.058060 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.053722    5051 scope.go:122] "RemoveContainer" containerID="5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef"
Nov 03 19:51:36.058060 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.053860    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"
Nov 03 19:51:36.083266 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.083233    5051 scope.go:122] "RemoveContainer" containerID="cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927"
Nov 03 19:51:36.098655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.098626    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"]
Nov 03 19:51:36.098798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.098671    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ef8f84-openshift-default-747457895-vprbb"]
Nov 03 19:51:36.099217 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.099203    5051 scope.go:122] "RemoveContainer" containerID="5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef"
Nov 03 19:51:36.099517 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:36.099497    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef\": container with ID starting with 5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef not found: ID does not exist" containerID="5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef"
Nov 03 19:51:36.099598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.099525    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef"} err="failed to get container status \"5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef\": rpc error: code = NotFound desc = could not find container \"5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef\": container with ID starting with 5408f959adbf14de3fdd3294e673b7e7e6c7e11698b990ce45f189a70a9a69ef not found: ID does not exist"
Nov 03 19:51:36.099598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.099544    5051 scope.go:122] "RemoveContainer" containerID="cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927"
Nov 03 19:51:36.099790 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:36.099772    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927\": container with ID starting with cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927 not found: ID does not exist" containerID="cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927"
Nov 03 19:51:36.099863 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.099793    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927"} err="failed to get container status \"cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927\": rpc error: code = NotFound desc = could not find container \"cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927\": container with ID starting with cff57a0f6abbdfa51f579ead0c6fd2438c11647fafa692cfddbf61b0b70d5927 not found: ID does not exist"
Nov 03 19:51:36.131008 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.130945    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:36.131008 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.130974    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:36.131008 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.130984    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2ae4144-2795-4fee-b8c7-9a5859c55d35-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:36.131008 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.130993    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz8qv\" (UniqueName: \"kubernetes.io/projected/e2ae4144-2795-4fee-b8c7-9a5859c55d35-kube-api-access-qz8qv\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:36.680397 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.680364    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:51:36.710536 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.710505    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"]
Nov 03 19:51:36.710958 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:36.710913    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy" containerID="cri-o://f06310fbc6fcc232bfa2b5feb47bcaa2ffa4b2b1c955fbc4b8b22cc44beaf599" gracePeriod=30
Nov 03 19:51:37.232940 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:37.232909    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" path="/var/lib/kubelet/pods/e2ae4144-2795-4fee-b8c7-9a5859c55d35/volumes"
Nov 03 19:51:37.970581 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:37.970543    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"]
Nov 03 19:51:37.982475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:37.982446    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"]
Nov 03 19:51:39.570068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:39.565068    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:51:40.015061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:40.014461    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:51:40.117066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:40.116435    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:51:40.151102 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:40.142975    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Nov 03 19:51:40.313256 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:40.313158    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 03 19:51:40.422084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:40.421439    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:51:40.427351 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:40.427324    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:51:40.470034 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:40.470006    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:51:41.019066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.018389    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Nov 03 19:51:41.019066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.019019    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:51:41.138066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.115169    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:51:41.138066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.115232    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:51:41.138066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.115623    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:51:41.164169 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.164120    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:51:41.189075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.172015    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:51:41.189075 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.173475    5051 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:51:41.319214 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.319123    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:51:41.324255 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.320930    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:51:41.327210 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.327184    5051 patch_prober.go:29] interesting pod/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 19:51:41.327381 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.327361    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 19:51:41.342172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.342142    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:51:41.425065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.424225    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:51:41.549221 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.442235    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:51:41.616121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.612228    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:51:41.762380 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.695796    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:51:41.789208 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.789178    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:51:41.789443 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:41.789429    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:51:41.937459 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:41.937248    5051 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eade3a5_b467_4599_b909_cb3306407caa.slice/crio-f06310fbc6fcc232bfa2b5feb47bcaa2ffa4b2b1c955fbc4b8b22cc44beaf599.scope\": RecentStats: unable to find data in memory cache]"
Nov 03 19:51:42.300355 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.299278    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"]
Nov 03 19:51:42.429815 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.429786    5051 generic.go:356] "Generic (PLEG): container finished" podID="8eade3a5-b467-4599-b909-cb3306407caa" containerID="f06310fbc6fcc232bfa2b5feb47bcaa2ffa4b2b1c955fbc4b8b22cc44beaf599" exitCode=0
Nov 03 19:51:42.430844 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.430781    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" event={"ID":"8eade3a5-b467-4599-b909-cb3306407caa","Type":"ContainerDied","Data":"f06310fbc6fcc232bfa2b5feb47bcaa2ffa4b2b1c955fbc4b8b22cc44beaf599"}
Nov 03 19:51:42.430968 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.430887    5051 scope.go:122] "RemoveContainer" containerID="7337569bc2f97c82612abf258988701a6ad700826c67d1c0007ebe7bfecb4bd8"
Nov 03 19:51:42.604088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.603400    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:51:42.682380 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.682293    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-credential-socket\") pod \"8eade3a5-b467-4599-b909-cb3306407caa\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") "
Nov 03 19:51:42.682958 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.682935    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8eade3a5-b467-4599-b909-cb3306407caa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8eade3a5-b467-4599-b909-cb3306407caa-istiod-ca-cert\") pod \"8eade3a5-b467-4599-b909-cb3306407caa\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") "
Nov 03 19:51:42.691365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.691344    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-envoy\") pod \"8eade3a5-b467-4599-b909-cb3306407caa\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") "
Nov 03 19:51:42.699524 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.699502    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-kube-api-access-gp26m\" (UniqueName: \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-kube-api-access-gp26m\") pod \"8eade3a5-b467-4599-b909-cb3306407caa\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") "
Nov 03 19:51:42.700082 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.700061    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-socket\") pod \"8eade3a5-b467-4599-b909-cb3306407caa\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") "
Nov 03 19:51:42.700523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.700503    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8eade3a5-b467-4599-b909-cb3306407caa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8eade3a5-b467-4599-b909-cb3306407caa-istio-podinfo\") pod \"8eade3a5-b467-4599-b909-cb3306407caa\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") "
Nov 03 19:51:42.700644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.700576    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-istio-token\" (UniqueName: \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-istio-token\") pod \"8eade3a5-b467-4599-b909-cb3306407caa\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") "
Nov 03 19:51:42.700644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.700607    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-certs\") pod \"8eade3a5-b467-4599-b909-cb3306407caa\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") "
Nov 03 19:51:42.700644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.700639    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-data\") pod \"8eade3a5-b467-4599-b909-cb3306407caa\" (UID: \"8eade3a5-b467-4599-b909-cb3306407caa\") "
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.682828    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-credential-socket" pod "8eade3a5-b467-4599-b909-cb3306407caa" (UID: "8eade3a5-b467-4599-b909-cb3306407caa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.691249    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eade3a5-b467-4599-b909-cb3306407caa-istiod-ca-cert" pod "8eade3a5-b467-4599-b909-cb3306407caa" (UID: "8eade3a5-b467-4599-b909-cb3306407caa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.700426    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-socket" pod "8eade3a5-b467-4599-b909-cb3306407caa" (UID: "8eade3a5-b467-4599-b909-cb3306407caa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703380    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-certs" pod "8eade3a5-b467-4599-b909-cb3306407caa" (UID: "8eade3a5-b467-4599-b909-cb3306407caa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.701714    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-kube-api-access-gp26m" pod "8eade3a5-b467-4599-b909-cb3306407caa" (UID: "8eade3a5-b467-4599-b909-cb3306407caa"). InnerVolumeSpecName "kube-api-access-gp26m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703169    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-envoy" pod "8eade3a5-b467-4599-b909-cb3306407caa" (UID: "8eade3a5-b467-4599-b909-cb3306407caa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703349    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8eade3a5-b467-4599-b909-cb3306407caa-istio-podinfo" pod "8eade3a5-b467-4599-b909-cb3306407caa" (UID: "8eade3a5-b467-4599-b909-cb3306407caa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703661    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-istio-token" pod "8eade3a5-b467-4599-b909-cb3306407caa" (UID: "8eade3a5-b467-4599-b909-cb3306407caa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703711    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-data" pod "8eade3a5-b467-4599-b909-cb3306407caa" (UID: "8eade3a5-b467-4599-b909-cb3306407caa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703853    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703871    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp26m\" (UniqueName: \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-kube-api-access-gp26m\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703881    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703890    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8eade3a5-b467-4599-b909-cb3306407caa-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703898    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8eade3a5-b467-4599-b909-cb3306407caa-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703906    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703914    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703923    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8eade3a5-b467-4599-b909-cb3306407caa-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:42.704076 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.703932    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8eade3a5-b467-4599-b909-cb3306407caa-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:42.972582 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.972532    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf" podUID="f742538a-ef9d-48d6-a4c6-db2d7c311797" containerName="payload-processing" containerID="cri-o://358006e1ab3389ee0485d5910000b3fad38fb6940441edc73a03c8f9c39be243" gracePeriod=25
Nov 03 19:51:42.984215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:42.984172    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld" podUID="36b180c7-1e40-49fd-81a0-c73f93567bbc" containerName="payload-pre-processing" containerID="cri-o://19cbe5c8d48fd663b25b0672c641fef0b8e25e2397f3c2465c779b55fb50d58c" gracePeriod=25
Nov 03 19:51:43.464275 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:51:43.464181    5051 logging.go:55] [core] [Channel #1422 SubChannel #1423]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.130:50052", ServerName: "10.132.0.130:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.130:50052: connect: connection refused"
Nov 03 19:51:43.470063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.465766    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq" event={"ID":"8eade3a5-b467-4599-b909-cb3306407caa","Type":"ContainerDied","Data":"793d1ccdbe145788a9a4cee72066d4d9bc76383ec7ef0879788bdb97987bb0fb"}
Nov 03 19:51:43.470063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.465849    5051 scope.go:122] "RemoveContainer" containerID="f06310fbc6fcc232bfa2b5feb47bcaa2ffa4b2b1c955fbc4b8b22cc44beaf599"
Nov 03 19:51:43.470063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.465964    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"
Nov 03 19:51:43.516758 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:51:43.477485    5051 logging.go:55] [core] [Channel #1424 SubChannel #1425]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.131:50052", ServerName: "10.132.0.131:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.131:50052: connect: connection refused"
Nov 03 19:51:43.520711 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.519667    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"]
Nov 03 19:51:43.520711 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.519851    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy" containerID="cri-o://c1442e341e4aaff70f0a0f757c955e45a6996fa5901e0b3ab4c883b1e5e23867" gracePeriod=30
Nov 03 19:51:43.523668 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.523601    5051 generic.go:356] "Generic (PLEG): container finished" podID="f742538a-ef9d-48d6-a4c6-db2d7c311797" containerID="358006e1ab3389ee0485d5910000b3fad38fb6940441edc73a03c8f9c39be243" exitCode=0
Nov 03 19:51:43.523975 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.523909    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf" event={"ID":"f742538a-ef9d-48d6-a4c6-db2d7c311797","Type":"ContainerDied","Data":"358006e1ab3389ee0485d5910000b3fad38fb6940441edc73a03c8f9c39be243"}
Nov 03 19:51:43.528568 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.528373    5051 generic.go:356] "Generic (PLEG): container finished" podID="36b180c7-1e40-49fd-81a0-c73f93567bbc" containerID="19cbe5c8d48fd663b25b0672c641fef0b8e25e2397f3c2465c779b55fb50d58c" exitCode=0
Nov 03 19:51:43.528568 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.528544    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld" event={"ID":"36b180c7-1e40-49fd-81a0-c73f93567bbc","Type":"ContainerDied","Data":"19cbe5c8d48fd663b25b0672c641fef0b8e25e2397f3c2465c779b55fb50d58c"}
Nov 03 19:51:43.836945 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.836912    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"]
Nov 03 19:51:43.837448 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.837413    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy" containerID="cri-o://14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3" gracePeriod=30
Nov 03 19:51:43.918651 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.918611    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"]
Nov 03 19:51:43.931148 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:43.931117    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-5444b0-openshift-default-588b84bfc-vxkxq"]
Nov 03 19:51:44.286066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.285634    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:51:44.286066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.285944    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:51:44.403847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.403760    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36b180c7-1e40-49fd-81a0-c73f93567bbc-kube-api-access-w6hp5\" (UniqueName: \"kubernetes.io/projected/36b180c7-1e40-49fd-81a0-c73f93567bbc-kube-api-access-w6hp5\") pod \"36b180c7-1e40-49fd-81a0-c73f93567bbc\" (UID: \"36b180c7-1e40-49fd-81a0-c73f93567bbc\") "
Nov 03 19:51:44.404271 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.404252    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36b180c7-1e40-49fd-81a0-c73f93567bbc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b180c7-1e40-49fd-81a0-c73f93567bbc-plugins-config-volume\") pod \"36b180c7-1e40-49fd-81a0-c73f93567bbc\" (UID: \"36b180c7-1e40-49fd-81a0-c73f93567bbc\") "
Nov 03 19:51:44.404422 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.404408    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f742538a-ef9d-48d6-a4c6-db2d7c311797-kube-api-access-lnj2j\" (UniqueName: \"kubernetes.io/projected/f742538a-ef9d-48d6-a4c6-db2d7c311797-kube-api-access-lnj2j\") pod \"f742538a-ef9d-48d6-a4c6-db2d7c311797\" (UID: \"f742538a-ef9d-48d6-a4c6-db2d7c311797\") "
Nov 03 19:51:44.404553 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.404540    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f742538a-ef9d-48d6-a4c6-db2d7c311797-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f742538a-ef9d-48d6-a4c6-db2d7c311797-plugins-config-volume\") pod \"f742538a-ef9d-48d6-a4c6-db2d7c311797\" (UID: \"f742538a-ef9d-48d6-a4c6-db2d7c311797\") "
Nov 03 19:51:44.404732 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.404703    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36b180c7-1e40-49fd-81a0-c73f93567bbc-plugins-config-volume" pod "36b180c7-1e40-49fd-81a0-c73f93567bbc" (UID: "36b180c7-1e40-49fd-81a0-c73f93567bbc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:44.405236 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.405204    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b180c7-1e40-49fd-81a0-c73f93567bbc-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:44.405462 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.405436    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f742538a-ef9d-48d6-a4c6-db2d7c311797-plugins-config-volume" pod "f742538a-ef9d-48d6-a4c6-db2d7c311797" (UID: "f742538a-ef9d-48d6-a4c6-db2d7c311797"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:44.407383 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.407349    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36b180c7-1e40-49fd-81a0-c73f93567bbc-kube-api-access-w6hp5" pod "36b180c7-1e40-49fd-81a0-c73f93567bbc" (UID: "36b180c7-1e40-49fd-81a0-c73f93567bbc"). InnerVolumeSpecName "kube-api-access-w6hp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:44.407770 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.407732    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f742538a-ef9d-48d6-a4c6-db2d7c311797-kube-api-access-lnj2j" pod "f742538a-ef9d-48d6-a4c6-db2d7c311797" (UID: "f742538a-ef9d-48d6-a4c6-db2d7c311797"). InnerVolumeSpecName "kube-api-access-lnj2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:44.467065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.464283    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld" podUID="36b180c7-1e40-49fd-81a0-c73f93567bbc" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.130:50052\" within 1s: context deadline exceeded"
Nov 03 19:51:44.467065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.464633    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf" podUID="f742538a-ef9d-48d6-a4c6-db2d7c311797" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.131:50052\" within 1s: context deadline exceeded"
Nov 03 19:51:44.506337 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.506304    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6hp5\" (UniqueName: \"kubernetes.io/projected/36b180c7-1e40-49fd-81a0-c73f93567bbc-kube-api-access-w6hp5\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:44.506337 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.506334    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnj2j\" (UniqueName: \"kubernetes.io/projected/f742538a-ef9d-48d6-a4c6-db2d7c311797-kube-api-access-lnj2j\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:44.506542 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.506351    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f742538a-ef9d-48d6-a4c6-db2d7c311797-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:44.597679 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.597637    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf" event={"ID":"f742538a-ef9d-48d6-a4c6-db2d7c311797","Type":"ContainerDied","Data":"b0fc7c928670e778137614ad880072c22de5acc28ebc5e9a95565f783ba1b888"}
Nov 03 19:51:44.597863 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.597700    5051 scope.go:122] "RemoveContainer" containerID="358006e1ab3389ee0485d5910000b3fad38fb6940441edc73a03c8f9c39be243"
Nov 03 19:51:44.598162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.598140    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"
Nov 03 19:51:44.602526 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.602491    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld" event={"ID":"36b180c7-1e40-49fd-81a0-c73f93567bbc","Type":"ContainerDied","Data":"92dad4097df6c9f327e570a7a0095fe21961d6a90fae1caa7ed5b72eadc945b3"}
Nov 03 19:51:44.602641 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.602600    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"
Nov 03 19:51:44.610370 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.610307    5051 scope.go:122] "RemoveContainer" containerID="19cbe5c8d48fd663b25b0672c641fef0b8e25e2397f3c2465c779b55fb50d58c"
Nov 03 19:51:44.636221 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.636195    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"]
Nov 03 19:51:44.639751 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.639729    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9ce34d-7c8fbc45bc-jvzqf"]
Nov 03 19:51:44.694981 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.694887    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"]
Nov 03 19:51:44.698087 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:44.698059    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9ce34d-94bddc499-vpfld"]
Nov 03 19:51:45.234186 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:45.234139    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36b180c7-1e40-49fd-81a0-c73f93567bbc" path="/var/lib/kubelet/pods/36b180c7-1e40-49fd-81a0-c73f93567bbc/volumes"
Nov 03 19:51:45.234970 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:45.234943    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8eade3a5-b467-4599-b909-cb3306407caa" path="/var/lib/kubelet/pods/8eade3a5-b467-4599-b909-cb3306407caa/volumes"
Nov 03 19:51:45.235613 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:45.235590    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f742538a-ef9d-48d6-a4c6-db2d7c311797" path="/var/lib/kubelet/pods/f742538a-ef9d-48d6-a4c6-db2d7c311797/volumes"
Nov 03 19:51:47.793771 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:47.793728    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"]
Nov 03 19:51:48.658085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.656281    5051 generic.go:356] "Generic (PLEG): container finished" podID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerID="c1442e341e4aaff70f0a0f757c955e45a6996fa5901e0b3ab4c883b1e5e23867" exitCode=0
Nov 03 19:51:48.658085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.656325    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" event={"ID":"425fcd7d-1420-41a8-89d4-503a9d259b89","Type":"ContainerDied","Data":"c1442e341e4aaff70f0a0f757c955e45a6996fa5901e0b3ab4c883b1e5e23867"}
Nov 03 19:51:48.658085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.656410    5051 scope.go:122] "RemoveContainer" containerID="96d8f4c3450d20f32bb5000ef953f7fd9e9e66ec16cb4849373169f4faebf089"
Nov 03 19:51:48.814181 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.814154    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:51:48.884544 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.884505    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-credential-socket\") pod \"425fcd7d-1420-41a8-89d4-503a9d259b89\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") "
Nov 03 19:51:48.884730 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.884593    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-kube-api-access-cckl7\" (UniqueName: \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-kube-api-access-cckl7\") pod \"425fcd7d-1420-41a8-89d4-503a9d259b89\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") "
Nov 03 19:51:48.884730 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.884672    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-certs\") pod \"425fcd7d-1420-41a8-89d4-503a9d259b89\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") "
Nov 03 19:51:48.884862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.884763    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/425fcd7d-1420-41a8-89d4-503a9d259b89-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/425fcd7d-1420-41a8-89d4-503a9d259b89-istiod-ca-cert\") pod \"425fcd7d-1420-41a8-89d4-503a9d259b89\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") "
Nov 03 19:51:48.884862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.884766    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-credential-socket" pod "425fcd7d-1420-41a8-89d4-503a9d259b89" (UID: "425fcd7d-1420-41a8-89d4-503a9d259b89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:48.884862 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.884808    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-socket\") pod \"425fcd7d-1420-41a8-89d4-503a9d259b89\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") "
Nov 03 19:51:48.885035 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.884891    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-podinfo\") pod \"425fcd7d-1420-41a8-89d4-503a9d259b89\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") "
Nov 03 19:51:48.885035 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.884977    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-envoy\") pod \"425fcd7d-1420-41a8-89d4-503a9d259b89\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") "
Nov 03 19:51:48.885035 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.884987    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-certs" pod "425fcd7d-1420-41a8-89d4-503a9d259b89" (UID: "425fcd7d-1420-41a8-89d4-503a9d259b89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:48.885035 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.885014    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-token\" (UniqueName: \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-token\") pod \"425fcd7d-1420-41a8-89d4-503a9d259b89\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") "
Nov 03 19:51:48.885289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.885092    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-data\") pod \"425fcd7d-1420-41a8-89d4-503a9d259b89\" (UID: \"425fcd7d-1420-41a8-89d4-503a9d259b89\") "
Nov 03 19:51:48.885289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.885218    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-socket" pod "425fcd7d-1420-41a8-89d4-503a9d259b89" (UID: "425fcd7d-1420-41a8-89d4-503a9d259b89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:48.885457 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.885433    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/425fcd7d-1420-41a8-89d4-503a9d259b89-istiod-ca-cert" pod "425fcd7d-1420-41a8-89d4-503a9d259b89" (UID: "425fcd7d-1420-41a8-89d4-503a9d259b89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:48.885892 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.885869    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:48.885985 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.885898    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:48.885985 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.885905    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-data" pod "425fcd7d-1420-41a8-89d4-503a9d259b89" (UID: "425fcd7d-1420-41a8-89d4-503a9d259b89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:48.885985 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.885914    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/425fcd7d-1420-41a8-89d4-503a9d259b89-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:48.885985 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.885927    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:48.888491 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.888467    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-token" pod "425fcd7d-1420-41a8-89d4-503a9d259b89" (UID: "425fcd7d-1420-41a8-89d4-503a9d259b89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:48.888588 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.888545    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-podinfo" pod "425fcd7d-1420-41a8-89d4-503a9d259b89" (UID: "425fcd7d-1420-41a8-89d4-503a9d259b89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:51:48.888775 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.888757    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-kube-api-access-cckl7" pod "425fcd7d-1420-41a8-89d4-503a9d259b89" (UID: "425fcd7d-1420-41a8-89d4-503a9d259b89"). InnerVolumeSpecName "kube-api-access-cckl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:48.898304 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.895968    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-envoy" pod "425fcd7d-1420-41a8-89d4-503a9d259b89" (UID: "425fcd7d-1420-41a8-89d4-503a9d259b89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:48.987421 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.987394    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:48.987421 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.987421    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:48.987587 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.987430    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cckl7\" (UniqueName: \"kubernetes.io/projected/425fcd7d-1420-41a8-89d4-503a9d259b89-kube-api-access-cckl7\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:48.987587 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.987441    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:48.987587 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:48.987450    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/425fcd7d-1420-41a8-89d4-503a9d259b89-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:49.442992 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.442968    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:51:49.446842 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.446815    5051 patch_prober.go:29] interesting pod/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Nov 03 19:51:49.446988 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.446862    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Nov 03 19:51:49.494794 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.494764    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-data\") pod \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") "
Nov 03 19:51:49.494958 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.494820    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istiod-ca-cert\") pod \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") "
Nov 03 19:51:49.494958 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.494882    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-certs\") pod \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") "
Nov 03 19:51:49.494958 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.494917    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-kube-api-access-wffl5\" (UniqueName: \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-kube-api-access-wffl5\") pod \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") "
Nov 03 19:51:49.495171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.494956    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-socket\") pod \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") "
Nov 03 19:51:49.495171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.495026    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-podinfo\") pod \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") "
Nov 03 19:51:49.495171 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.495099    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-data" pod "4b0a399e-8e57-4d85-bdb5-94577f83a19c" (UID: "4b0a399e-8e57-4d85-bdb5-94577f83a19c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:49.495337 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.495225    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-envoy\") pod \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") "
Nov 03 19:51:49.495337 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.495245    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-certs" pod "4b0a399e-8e57-4d85-bdb5-94577f83a19c" (UID: "4b0a399e-8e57-4d85-bdb5-94577f83a19c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:49.495337 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.495328    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-token\" (UniqueName: \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-token\") pod \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") "
Nov 03 19:51:49.495525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.495396    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-credential-socket\") pod \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\" (UID: \"4b0a399e-8e57-4d85-bdb5-94577f83a19c\") "
Nov 03 19:51:49.495525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.495406    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-socket" pod "4b0a399e-8e57-4d85-bdb5-94577f83a19c" (UID: "4b0a399e-8e57-4d85-bdb5-94577f83a19c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:49.495525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.495334    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istiod-ca-cert" pod "4b0a399e-8e57-4d85-bdb5-94577f83a19c" (UID: "4b0a399e-8e57-4d85-bdb5-94577f83a19c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:49.495928 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.495901    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-credential-socket" pod "4b0a399e-8e57-4d85-bdb5-94577f83a19c" (UID: "4b0a399e-8e57-4d85-bdb5-94577f83a19c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:49.496352 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.496326    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:49.496475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.496354    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:49.496475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.496369    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:49.496475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.496383    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:49.496475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.496399    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:49.497737 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.497713    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-kube-api-access-wffl5" pod "4b0a399e-8e57-4d85-bdb5-94577f83a19c" (UID: "4b0a399e-8e57-4d85-bdb5-94577f83a19c"). InnerVolumeSpecName "kube-api-access-wffl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:49.497950 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.497924    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-envoy" pod "4b0a399e-8e57-4d85-bdb5-94577f83a19c" (UID: "4b0a399e-8e57-4d85-bdb5-94577f83a19c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:49.497950 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.497934    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-token" pod "4b0a399e-8e57-4d85-bdb5-94577f83a19c" (UID: "4b0a399e-8e57-4d85-bdb5-94577f83a19c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:49.498426 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.498407    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-podinfo" pod "4b0a399e-8e57-4d85-bdb5-94577f83a19c" (UID: "4b0a399e-8e57-4d85-bdb5-94577f83a19c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:51:49.599361 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.599329    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wffl5\" (UniqueName: \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-kube-api-access-wffl5\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:49.599521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.599386    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:49.599521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.599403    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:49.599521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.599417    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b0a399e-8e57-4d85-bdb5-94577f83a19c-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:49.666703 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.666655    5051 generic.go:356] "Generic (PLEG): container finished" podID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerID="a1e47d98d2349afc93b60d10631ea08a066de639c970ee387ce5c2f459482c00" exitCode=0
Nov 03 19:51:49.666888 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.666835    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" event={"ID":"4761a958-3fed-4c9f-aa46-314065cebb4a","Type":"ContainerDied","Data":"a1e47d98d2349afc93b60d10631ea08a066de639c970ee387ce5c2f459482c00"}
Nov 03 19:51:49.667520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.667379    5051 scope.go:122] "RemoveContainer" containerID="a1e47d98d2349afc93b60d10631ea08a066de639c970ee387ce5c2f459482c00"
Nov 03 19:51:49.668695 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.668671    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"
Nov 03 19:51:49.668798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.668704    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g" event={"ID":"425fcd7d-1420-41a8-89d4-503a9d259b89","Type":"ContainerDied","Data":"4ef46fa22f72620a7ece283768366b93f791802c5e0b8c990e45d044077ae957"}
Nov 03 19:51:49.668798 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.668751    5051 scope.go:122] "RemoveContainer" containerID="c1442e341e4aaff70f0a0f757c955e45a6996fa5901e0b3ab4c883b1e5e23867"
Nov 03 19:51:49.669267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.669214    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:49.669373 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.669306    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:49.669373 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.669356    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:49.675453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.674901    5051 generic.go:356] "Generic (PLEG): container finished" podID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerID="14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3" exitCode=0
Nov 03 19:51:49.675453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.674974    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"
Nov 03 19:51:49.675453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.675005    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" event={"ID":"4b0a399e-8e57-4d85-bdb5-94577f83a19c","Type":"ContainerDied","Data":"14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3"}
Nov 03 19:51:49.675453 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.675056    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc" event={"ID":"4b0a399e-8e57-4d85-bdb5-94577f83a19c","Type":"ContainerDied","Data":"bd3504c89b414f1a5ced873466e08566b3308fd10c3d20262fc23eb4af8a5698"}
Nov 03 19:51:49.685697 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.685637    5051 generic.go:356] "Generic (PLEG): container finished" podID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerID="9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0" exitCode=0
Nov 03 19:51:49.685697 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.685676    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" event={"ID":"7a258474-8531-448f-8e4a-b7e4b39728fc","Type":"ContainerDied","Data":"9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0"}
Nov 03 19:51:49.686151 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.686130    5051 scope.go:122] "RemoveContainer" containerID="9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0"
Nov 03 19:51:49.697144 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.694242    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:49.697144 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.694331    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:49.697144 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.694377    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:51:49.703522 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.703491    5051 scope.go:122] "RemoveContainer" containerID="14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3"
Nov 03 19:51:49.743890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.743865    5051 scope.go:122] "RemoveContainer" containerID="d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4"
Nov 03 19:51:49.755276 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.754693    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"]
Nov 03 19:51:49.759373 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.759343    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-300831-openshift-default-656d95d487-xtm2g"]
Nov 03 19:51:49.776289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.774973    5051 scope.go:122] "RemoveContainer" containerID="14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3"
Nov 03 19:51:49.780690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.780662    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"]
Nov 03 19:51:49.783257 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:49.783228    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3\": container with ID starting with 14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3 not found: ID does not exist" containerID="14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3"
Nov 03 19:51:49.783423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.783399    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3"} err="failed to get container status \"14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3\": rpc error: code = NotFound desc = could not find container \"14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3\": container with ID starting with 14f01256eea0435c25f06b3b57c9e1eeac57ebf3febe15e0483e361f698a85d3 not found: ID does not exist"
Nov 03 19:51:49.783523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.783512    5051 scope.go:122] "RemoveContainer" containerID="d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4"
Nov 03 19:51:49.784015 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:49.783991    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4\": container with ID starting with d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4 not found: ID does not exist" containerID="d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4"
Nov 03 19:51:49.784261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.784209    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4"} err="failed to get container status \"d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4\": rpc error: code = NotFound desc = could not find container \"d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4\": container with ID starting with d9c0f909e9cab31fd0728481f28e2b7eb3dcc24a06b33835f64299f01978fdb4 not found: ID does not exist"
Nov 03 19:51:49.784666 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:49.784649    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-8cff38-openshift-default-7d675cbb7-v59jc"]
Nov 03 19:51:50.701581 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:50.701535    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" event={"ID":"7a258474-8531-448f-8e4a-b7e4b39728fc","Type":"ContainerStarted","Data":"5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808"}
Nov 03 19:51:50.706544 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:50.706514    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" event={"ID":"4761a958-3fed-4c9f-aa46-314065cebb4a","Type":"ContainerStarted","Data":"839aec472d8840e31cc8b15ee4ba27c32d084af4c6c06577cfded2ab02a10348"}
Nov 03 19:51:50.964063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:50.960492    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:51:50.964063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:50.960623    5051 patch_prober.go:29] interesting pod/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Nov 03 19:51:50.964063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:50.960666    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Nov 03 19:51:51.231690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:51.231605    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" path="/var/lib/kubelet/pods/425fcd7d-1420-41a8-89d4-503a9d259b89/volumes"
Nov 03 19:51:51.232202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:51.232177    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" path="/var/lib/kubelet/pods/4b0a399e-8e57-4d85-bdb5-94577f83a19c/volumes"
Nov 03 19:51:51.481065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:51.480403    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:51:51.481065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:51.480529    5051 patch_prober.go:29] interesting pod/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Nov 03 19:51:51.481065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:51.480574    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Nov 03 19:51:51.960494 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:51.960456    5051 patch_prober.go:29] interesting pod/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Nov 03 19:51:51.961135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:51.961102    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Nov 03 19:51:52.480489 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:52.480447    5051 patch_prober.go:29] interesting pod/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Nov 03 19:51:52.480789 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:52.480760    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Nov 03 19:51:52.794186 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:52.794079    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m" podUID="82b74e21-57a7-489f-abe4-9814cf683aab" containerName="payload-pre-processing" containerID="cri-o://92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e" gracePeriod=25
Nov 03 19:51:52.961065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:52.956419    5051 patch_prober.go:29] interesting pod/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Nov 03 19:51:52.961065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:52.956479    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Nov 03 19:51:53.230319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.230293    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:51:53.413528 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.413492    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82b74e21-57a7-489f-abe4-9814cf683aab-kube-api-access-kvdpc\" (UniqueName: \"kubernetes.io/projected/82b74e21-57a7-489f-abe4-9814cf683aab-kube-api-access-kvdpc\") pod \"82b74e21-57a7-489f-abe4-9814cf683aab\" (UID: \"82b74e21-57a7-489f-abe4-9814cf683aab\") "
Nov 03 19:51:53.413718 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.413676    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82b74e21-57a7-489f-abe4-9814cf683aab-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b74e21-57a7-489f-abe4-9814cf683aab-plugins-config-volume\") pod \"82b74e21-57a7-489f-abe4-9814cf683aab\" (UID: \"82b74e21-57a7-489f-abe4-9814cf683aab\") "
Nov 03 19:51:53.414708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.414678    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82b74e21-57a7-489f-abe4-9814cf683aab-plugins-config-volume" pod "82b74e21-57a7-489f-abe4-9814cf683aab" (UID: "82b74e21-57a7-489f-abe4-9814cf683aab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:53.417726 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.417693    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b74e21-57a7-489f-abe4-9814cf683aab-kube-api-access-kvdpc" pod "82b74e21-57a7-489f-abe4-9814cf683aab" (UID: "82b74e21-57a7-489f-abe4-9814cf683aab"). InnerVolumeSpecName "kube-api-access-kvdpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:53.491498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.491456    5051 patch_prober.go:29] interesting pod/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Nov 03 19:51:53.491708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.491523    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Nov 03 19:51:53.516393 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.516363    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvdpc\" (UniqueName: \"kubernetes.io/projected/82b74e21-57a7-489f-abe4-9814cf683aab-kube-api-access-kvdpc\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:53.516393 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.516395    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b74e21-57a7-489f-abe4-9814cf683aab-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:53.738985 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.738948    5051 generic.go:356] "Generic (PLEG): container finished" podID="82b74e21-57a7-489f-abe4-9814cf683aab" containerID="92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e" exitCode=0
Nov 03 19:51:53.739484 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.739452    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m" event={"ID":"82b74e21-57a7-489f-abe4-9814cf683aab","Type":"ContainerDied","Data":"92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e"}
Nov 03 19:51:53.739696 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.739674    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m" event={"ID":"82b74e21-57a7-489f-abe4-9814cf683aab","Type":"ContainerDied","Data":"52b459e97f796046fd0d6dcc52eccaf47f36eecf85bf08a8a355bfbad6d6477b"}
Nov 03 19:51:53.739879 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.739852    5051 scope.go:122] "RemoveContainer" containerID="92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e"
Nov 03 19:51:53.753268 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.753184    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"
Nov 03 19:51:53.816328 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.816138    5051 scope.go:122] "RemoveContainer" containerID="92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e"
Nov 03 19:51:53.816528 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:53.816508    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e\": container with ID starting with 92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e not found: ID does not exist" containerID="92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e"
Nov 03 19:51:53.816593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.816536    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e"} err="failed to get container status \"92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e\": rpc error: code = NotFound desc = could not find container \"92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e\": container with ID starting with 92077091eb482252fd9abb4153293b1d3004227383b65fd9da0589fdc8429e8e not found: ID does not exist"
Nov 03 19:51:53.828663 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.828635    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"]
Nov 03 19:51:53.831629 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.831603    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2c3fc4-9f8bc97bc-7hv9m"]
Nov 03 19:51:53.959490 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.959452    5051 patch_prober.go:29] interesting pod/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Nov 03 19:51:53.960189 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:53.959774    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Nov 03 19:51:54.309179 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:54.309146    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"]
Nov 03 19:51:54.495818 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:54.495535    5051 patch_prober.go:29] interesting pod/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Nov 03 19:51:54.495818 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:54.495599    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Nov 03 19:51:54.975880 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:54.975841    5051 patch_prober.go:29] interesting pod/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Nov 03 19:51:54.983871 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:54.983831    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Nov 03 19:51:55.044066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:55.040554    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"]
Nov 03 19:51:55.044066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:55.040830    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b" podUID="98ab7b99-b947-4052-81f6-dfe33dd18191" containerName="maas-api" containerID="cri-o://2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f" gracePeriod=30
Nov 03 19:51:55.250115 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:55.239207    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82b74e21-57a7-489f-abe4-9814cf683aab" path="/var/lib/kubelet/pods/82b74e21-57a7-489f-abe4-9814cf683aab/volumes"
Nov 03 19:51:55.481066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:55.480078    5051 patch_prober.go:29] interesting pod/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Nov 03 19:51:55.481066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:55.480157    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Nov 03 19:51:55.964013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:55.963979    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:51:55.968885 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:55.968836    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-metrics-certs\" (UniqueName: \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-metrics-certs\") pod \"98ab7b99-b947-4052-81f6-dfe33dd18191\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") "
Nov 03 19:51:55.977549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:55.977518    5051 patch_prober.go:29] interesting pod/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Nov 03 19:51:55.977789 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:55.977761    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Nov 03 19:51:56.095114 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.091360    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98ab7b99-b947-4052-81f6-dfe33dd18191-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98ab7b99-b947-4052-81f6-dfe33dd18191-openshift-service-ca\") pod \"98ab7b99-b947-4052-81f6-dfe33dd18191\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") "
Nov 03 19:51:56.100064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.096056    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-maas-api-tls\") pod \"98ab7b99-b947-4052-81f6-dfe33dd18191\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") "
Nov 03 19:51:56.107069 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.100407    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98ab7b99-b947-4052-81f6-dfe33dd18191-kube-api-access-cnvwb\" (UniqueName: \"kubernetes.io/projected/98ab7b99-b947-4052-81f6-dfe33dd18191-kube-api-access-cnvwb\") pod \"98ab7b99-b947-4052-81f6-dfe33dd18191\" (UID: \"98ab7b99-b947-4052-81f6-dfe33dd18191\") "
Nov 03 19:51:56.146142 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.098266    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98ab7b99-b947-4052-81f6-dfe33dd18191-openshift-service-ca" pod "98ab7b99-b947-4052-81f6-dfe33dd18191" (UID: "98ab7b99-b947-4052-81f6-dfe33dd18191"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:56.146142 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.139664    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-maas-api-tls" pod "98ab7b99-b947-4052-81f6-dfe33dd18191" (UID: "98ab7b99-b947-4052-81f6-dfe33dd18191"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:56.146142 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.139715    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-metrics-certs" pod "98ab7b99-b947-4052-81f6-dfe33dd18191" (UID: "98ab7b99-b947-4052-81f6-dfe33dd18191"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:56.149034 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.149005    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:56.302202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.275335    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98ab7b99-b947-4052-81f6-dfe33dd18191-kube-api-access-cnvwb" pod "98ab7b99-b947-4052-81f6-dfe33dd18191" (UID: "98ab7b99-b947-4052-81f6-dfe33dd18191"). InnerVolumeSpecName "kube-api-access-cnvwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:56.302202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.302104    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98ab7b99-b947-4052-81f6-dfe33dd18191-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:56.302202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.302169    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98ab7b99-b947-4052-81f6-dfe33dd18191-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:56.362155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.359141    5051 generic.go:356] "Generic (PLEG): container finished" podID="98ab7b99-b947-4052-81f6-dfe33dd18191" containerID="2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f" exitCode=0
Nov 03 19:51:56.362155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.359362    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b" event={"ID":"98ab7b99-b947-4052-81f6-dfe33dd18191","Type":"ContainerDied","Data":"2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f"}
Nov 03 19:51:56.362155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.359417    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b" event={"ID":"98ab7b99-b947-4052-81f6-dfe33dd18191","Type":"ContainerDied","Data":"aa1f20e30de328deb1a57b7601a8620e7e8c9a9c5905cb16b02ab370402256ab"}
Nov 03 19:51:56.362155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.359448    5051 scope.go:122] "RemoveContainer" containerID="2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f"
Nov 03 19:51:56.362155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.359588    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"
Nov 03 19:51:56.417159 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.412994    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnvwb\" (UniqueName: \"kubernetes.io/projected/98ab7b99-b947-4052-81f6-dfe33dd18191-kube-api-access-cnvwb\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:56.480130 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.480094    5051 patch_prober.go:29] interesting pod/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Nov 03 19:51:56.480413 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.480382    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Nov 03 19:51:56.519064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.506129    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"]
Nov 03 19:51:56.523467 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.523436    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29ab98-68cdbdb465-brw8b"]
Nov 03 19:51:56.526135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.526112    5051 scope.go:122] "RemoveContainer" containerID="2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f"
Nov 03 19:51:56.528109 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:56.526942    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f\": container with ID starting with 2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f not found: ID does not exist" containerID="2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f"
Nov 03 19:51:56.528109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.526984    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f"} err="failed to get container status \"2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f\": rpc error: code = NotFound desc = could not find container \"2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f\": container with ID starting with 2b4dbce085bb787044966592a314f618c15cafb769de6bccbf68e7e6c0bc699f not found: ID does not exist"
Nov 03 19:51:56.972342 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.972313    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:51:56.978244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.978091    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:51:56.980638 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:56.980569    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:51:57.229717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:57.229627    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98ab7b99-b947-4052-81f6-dfe33dd18191" path="/var/lib/kubelet/pods/98ab7b99-b947-4052-81f6-dfe33dd18191/volumes"
Nov 03 19:51:57.483364 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:57.483289    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:51:57.483703 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:57.483678    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:51:57.484303 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:57.484285    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:51:58.048133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:58.048097    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"]
Nov 03 19:51:58.076200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:58.076167    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"]
Nov 03 19:51:58.077951 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:58.076475    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerName="istio-proxy" containerID="cri-o://2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a" gracePeriod=30
Nov 03 19:51:58.077951 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:58.076537    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89" podUID="dab38bfe-c49e-4031-8391-6075964ace09" containerName="istio-proxy" containerID="cri-o://72049b7fe571b009faf919b1c3b84d0606c1f6a75526a924aa42173a1a8d558e" gracePeriod=30
Nov 03 19:51:58.832737 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:58.832694    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main" containerID="cri-o://fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385" gracePeriod=45
Nov 03 19:51:59.179236 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.179210    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:51:59.281101 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281072    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-home\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-home\") pod \"8d11a037-b776-43f9-9d65-fa507737d381\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") "
Nov 03 19:51:59.281270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281136    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8d11a037-b776-43f9-9d65-fa507737d381-tls-certs\" (UniqueName: \"kubernetes.io/secret/8d11a037-b776-43f9-9d65-fa507737d381-tls-certs\") pod \"8d11a037-b776-43f9-9d65-fa507737d381\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") "
Nov 03 19:51:59.281270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281174    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d11a037-b776-43f9-9d65-fa507737d381-kube-api-access-p6ftf\" (UniqueName: \"kubernetes.io/projected/8d11a037-b776-43f9-9d65-fa507737d381-kube-api-access-p6ftf\") pod \"8d11a037-b776-43f9-9d65-fa507737d381\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") "
Nov 03 19:51:59.281270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281218    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-model-cache\") pod \"8d11a037-b776-43f9-9d65-fa507737d381\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") "
Nov 03 19:51:59.281443 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281305    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-tmp-dir\") pod \"8d11a037-b776-43f9-9d65-fa507737d381\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") "
Nov 03 19:51:59.281443 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281338    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-dshm\") pod \"8d11a037-b776-43f9-9d65-fa507737d381\" (UID: \"8d11a037-b776-43f9-9d65-fa507737d381\") "
Nov 03 19:51:59.281443 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281376    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-home" pod "8d11a037-b776-43f9-9d65-fa507737d381" (UID: "8d11a037-b776-43f9-9d65-fa507737d381"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:59.281613 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281574    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-model-cache" pod "8d11a037-b776-43f9-9d65-fa507737d381" (UID: "8d11a037-b776-43f9-9d65-fa507737d381"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:59.281613 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281602    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-tmp-dir" pod "8d11a037-b776-43f9-9d65-fa507737d381" (UID: "8d11a037-b776-43f9-9d65-fa507737d381"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:59.281841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281824    5051 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-home\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:59.281903 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281843    5051 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-model-cache\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:59.281903 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.281854    5051 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-tmp-dir\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:59.283744 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.283717    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-dshm" pod "8d11a037-b776-43f9-9d65-fa507737d381" (UID: "8d11a037-b776-43f9-9d65-fa507737d381"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:51:59.284056 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.284024    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d11a037-b776-43f9-9d65-fa507737d381-tls-certs" pod "8d11a037-b776-43f9-9d65-fa507737d381" (UID: "8d11a037-b776-43f9-9d65-fa507737d381"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:51:59.284158 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.284086    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d11a037-b776-43f9-9d65-fa507737d381-kube-api-access-p6ftf" pod "8d11a037-b776-43f9-9d65-fa507737d381" (UID: "8d11a037-b776-43f9-9d65-fa507737d381"). InnerVolumeSpecName "kube-api-access-p6ftf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:59.309898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.309823    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w" podUID="0e1ad4ab-300a-425c-b752-bd6dd6108c4a" containerName="payload-pre-processing" containerID="cri-o://ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5" gracePeriod=25
Nov 03 19:51:59.388205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.388128    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"]
Nov 03 19:51:59.388613 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.388583    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf" podUID="1cf13e8c-1976-4218-943b-4deaf2dde7eb" containerName="maas-api" containerID="cri-o://6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2" gracePeriod=30
Nov 03 19:51:59.390105 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.390087    5051 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d11a037-b776-43f9-9d65-fa507737d381-tls-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:59.390244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.390230    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6ftf\" (UniqueName: \"kubernetes.io/projected/8d11a037-b776-43f9-9d65-fa507737d381-kube-api-access-p6ftf\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:59.390340 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.390327    5051 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d11a037-b776-43f9-9d65-fa507737d381-dshm\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:59.405791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.405763    5051 generic.go:356] "Generic (PLEG): container finished" podID="8d11a037-b776-43f9-9d65-fa507737d381" containerID="fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385" exitCode=0
Nov 03 19:51:59.405907 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.405823    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"
Nov 03 19:51:59.405907 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.405825    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" event={"ID":"8d11a037-b776-43f9-9d65-fa507737d381","Type":"ContainerDied","Data":"fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385"}
Nov 03 19:51:59.406025 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.405937    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl" event={"ID":"8d11a037-b776-43f9-9d65-fa507737d381","Type":"ContainerDied","Data":"2331981ddfa901b741a6bf5a082db03e053df0939107eb400b01539eb1ad04f1"}
Nov 03 19:51:59.406025 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.405964    5051 scope.go:122] "RemoveContainer" containerID="fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385"
Nov 03 19:51:59.483252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.483221    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"]
Nov 03 19:51:59.487735 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.487709    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-2c3fc4/distinct-w1-2c3fc4-kserve-8558d794c7-wtqpl"]
Nov 03 19:51:59.510296 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.510273    5051 scope.go:122] "RemoveContainer" containerID="fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385"
Nov 03 19:51:59.510690 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:51:59.510664    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385\": container with ID starting with fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385 not found: ID does not exist" containerID="fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385"
Nov 03 19:51:59.510759 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.510730    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385"} err="failed to get container status \"fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385\": rpc error: code = NotFound desc = could not find container \"fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385\": container with ID starting with fce70cc761cecfb15df369e254ff6bbdda0b45b696bb8ef7b66ed23b4173a385 not found: ID does not exist"
Nov 03 19:51:59.519226 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:51:59.519203    5051 logging.go:55] [core] [Channel #1438 SubChannel #1439]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.134:50052", ServerName: "10.132.0.134:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.134:50052: connect: connection refused"
Nov 03 19:51:59.788657 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.784683    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:51:59.914661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.914632    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-kube-api-access-djzzt\" (UniqueName: \"kubernetes.io/projected/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-kube-api-access-djzzt\") pod \"0e1ad4ab-300a-425c-b752-bd6dd6108c4a\" (UID: \"0e1ad4ab-300a-425c-b752-bd6dd6108c4a\") "
Nov 03 19:51:59.914852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.914827    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-plugins-config-volume\") pod \"0e1ad4ab-300a-425c-b752-bd6dd6108c4a\" (UID: \"0e1ad4ab-300a-425c-b752-bd6dd6108c4a\") "
Nov 03 19:51:59.915215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.915187    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-plugins-config-volume" pod "0e1ad4ab-300a-425c-b752-bd6dd6108c4a" (UID: "0e1ad4ab-300a-425c-b752-bd6dd6108c4a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:51:59.915425 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.915375    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:51:59.916775 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.916755    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-kube-api-access-djzzt" pod "0e1ad4ab-300a-425c-b752-bd6dd6108c4a" (UID: "0e1ad4ab-300a-425c-b752-bd6dd6108c4a"). InnerVolumeSpecName "kube-api-access-djzzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:51:59.917632 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:51:59.917613    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:52:00.016379 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.016351    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-metrics-certs\") pod \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") "
Nov 03 19:52:00.016525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.016391    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1cf13e8c-1976-4218-943b-4deaf2dde7eb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cf13e8c-1976-4218-943b-4deaf2dde7eb-openshift-service-ca\") pod \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") "
Nov 03 19:52:00.016525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.016515    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cf13e8c-1976-4218-943b-4deaf2dde7eb-kube-api-access-7hblh\" (UniqueName: \"kubernetes.io/projected/1cf13e8c-1976-4218-943b-4deaf2dde7eb-kube-api-access-7hblh\") pod \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") "
Nov 03 19:52:00.016616 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.016560    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-maas-api-tls\") pod \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\" (UID: \"1cf13e8c-1976-4218-943b-4deaf2dde7eb\") "
Nov 03 19:52:00.016997 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.016785    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cf13e8c-1976-4218-943b-4deaf2dde7eb-openshift-service-ca" pod "1cf13e8c-1976-4218-943b-4deaf2dde7eb" (UID: "1cf13e8c-1976-4218-943b-4deaf2dde7eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:00.017150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.017012    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djzzt\" (UniqueName: \"kubernetes.io/projected/0e1ad4ab-300a-425c-b752-bd6dd6108c4a-kube-api-access-djzzt\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:00.017150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.017034    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cf13e8c-1976-4218-943b-4deaf2dde7eb-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:00.019088 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.019034    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-maas-api-tls" pod "1cf13e8c-1976-4218-943b-4deaf2dde7eb" (UID: "1cf13e8c-1976-4218-943b-4deaf2dde7eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:00.019205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.019117    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-metrics-certs" pod "1cf13e8c-1976-4218-943b-4deaf2dde7eb" (UID: "1cf13e8c-1976-4218-943b-4deaf2dde7eb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:00.019205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.019147    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cf13e8c-1976-4218-943b-4deaf2dde7eb-kube-api-access-7hblh" pod "1cf13e8c-1976-4218-943b-4deaf2dde7eb" (UID: "1cf13e8c-1976-4218-943b-4deaf2dde7eb"). InnerVolumeSpecName "kube-api-access-7hblh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:00.118363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.118308    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:00.118363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.118348    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cf13e8c-1976-4218-943b-4deaf2dde7eb-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:00.118363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.118364    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hblh\" (UniqueName: \"kubernetes.io/projected/1cf13e8c-1976-4218-943b-4deaf2dde7eb-kube-api-access-7hblh\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:00.414133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.414055    5051 generic.go:356] "Generic (PLEG): container finished" podID="0e1ad4ab-300a-425c-b752-bd6dd6108c4a" containerID="ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5" exitCode=0
Nov 03 19:52:00.414133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.414109    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"
Nov 03 19:52:00.414619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.414136    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w" event={"ID":"0e1ad4ab-300a-425c-b752-bd6dd6108c4a","Type":"ContainerDied","Data":"ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5"}
Nov 03 19:52:00.414619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.414184    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w" event={"ID":"0e1ad4ab-300a-425c-b752-bd6dd6108c4a","Type":"ContainerDied","Data":"81991c7e00442b50a9f45c02a7d76c65dbf5bbf6c36c2794a10d7edb6a4a5403"}
Nov 03 19:52:00.414619 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.414210    5051 scope.go:122] "RemoveContainer" containerID="ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5"
Nov 03 19:52:00.415690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.415663    5051 generic.go:356] "Generic (PLEG): container finished" podID="1cf13e8c-1976-4218-943b-4deaf2dde7eb" containerID="6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2" exitCode=0
Nov 03 19:52:00.415808 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.415692    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf" event={"ID":"1cf13e8c-1976-4218-943b-4deaf2dde7eb","Type":"ContainerDied","Data":"6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2"}
Nov 03 19:52:00.415808 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.415713    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"
Nov 03 19:52:00.415808 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.415736    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf" event={"ID":"1cf13e8c-1976-4218-943b-4deaf2dde7eb","Type":"ContainerDied","Data":"7cabc386825c2979d317dc6276c981fc4e43816f663b3a677906b5c00ddbf38d"}
Nov 03 19:52:00.426960 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.426911    5051 scope.go:122] "RemoveContainer" containerID="ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5"
Nov 03 19:52:00.427279 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:00.427256    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5\": container with ID starting with ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5 not found: ID does not exist" containerID="ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5"
Nov 03 19:52:00.427382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.427284    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5"} err="failed to get container status \"ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5\": rpc error: code = NotFound desc = could not find container \"ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5\": container with ID starting with ff1974aef8c61c14c4a612407d6d3c4b949e78fa82ff18e993d5cdcf6f6028f5 not found: ID does not exist"
Nov 03 19:52:00.427382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.427300    5051 scope.go:122] "RemoveContainer" containerID="6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2"
Nov 03 19:52:00.437141 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.437115    5051 scope.go:122] "RemoveContainer" containerID="6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2"
Nov 03 19:52:00.437409 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:00.437390    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2\": container with ID starting with 6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2 not found: ID does not exist" containerID="6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2"
Nov 03 19:52:00.437482 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.437418    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2"} err="failed to get container status \"6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2\": rpc error: code = NotFound desc = could not find container \"6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2\": container with ID starting with 6fbe414d49cd42fc08a82298d9809e471eb6449e05dc8c82b32be4108bf054a2 not found: ID does not exist"
Nov 03 19:52:00.466882 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.466845    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"]
Nov 03 19:52:00.469822 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.469798    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w"]
Nov 03 19:52:00.481606 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.481583    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"]
Nov 03 19:52:00.484267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.484245    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9ce34d-cc8d97cc-4l2nf"]
Nov 03 19:52:00.517685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:00.517652    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-b-ab584a-86d766486c-8f82w" podUID="0e1ad4ab-300a-425c-b752-bd6dd6108c4a" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.134:50052\" within 1s: context deadline exceeded"
Nov 03 19:52:01.229802 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:01.229767    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e1ad4ab-300a-425c-b752-bd6dd6108c4a" path="/var/lib/kubelet/pods/0e1ad4ab-300a-425c-b752-bd6dd6108c4a/volumes"
Nov 03 19:52:01.230288 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:01.230271    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1cf13e8c-1976-4218-943b-4deaf2dde7eb" path="/var/lib/kubelet/pods/1cf13e8c-1976-4218-943b-4deaf2dde7eb/volumes"
Nov 03 19:52:01.230724 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:01.230710    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d11a037-b776-43f9-9d65-fa507737d381" path="/var/lib/kubelet/pods/8d11a037-b776-43f9-9d65-fa507737d381/volumes"
Nov 03 19:52:02.769951 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:02.769913    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:52:03.394327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.394301    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:52:03.444780 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.444744    5051 generic.go:356] "Generic (PLEG): container finished" podID="dab38bfe-c49e-4031-8391-6075964ace09" containerID="72049b7fe571b009faf919b1c3b84d0606c1f6a75526a924aa42173a1a8d558e" exitCode=0
Nov 03 19:52:03.444967 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.444840    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89" event={"ID":"dab38bfe-c49e-4031-8391-6075964ace09","Type":"ContainerDied","Data":"72049b7fe571b009faf919b1c3b84d0606c1f6a75526a924aa42173a1a8d558e"}
Nov 03 19:52:03.446482 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.446458    5051 generic.go:356] "Generic (PLEG): container finished" podID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerID="2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a" exitCode=0
Nov 03 19:52:03.446613 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.446507    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"
Nov 03 19:52:03.446613 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.446522    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" event={"ID":"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8","Type":"ContainerDied","Data":"2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a"}
Nov 03 19:52:03.446613 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.446578    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf" event={"ID":"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8","Type":"ContainerDied","Data":"339f2ee109067acdaf272165fff32259e2d843e828a6fa161457880b59143792"}
Nov 03 19:52:03.446613 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.446610    5051 scope.go:122] "RemoveContainer" containerID="2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a"
Nov 03 19:52:03.456914 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.456868    5051 scope.go:122] "RemoveContainer" containerID="2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a"
Nov 03 19:52:03.457238 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:03.457212    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a\": container with ID starting with 2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a not found: ID does not exist" containerID="2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a"
Nov 03 19:52:03.457310 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.457251    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a"} err="failed to get container status \"2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a\": rpc error: code = NotFound desc = could not find container \"2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a\": container with ID starting with 2ea0578909b88f88c1828d272a46ae785e4fddfa199f09ecdf0787e17cdd089a not found: ID does not exist"
Nov 03 19:52:03.475169 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.475066    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-credential-socket\") pod \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") "
Nov 03 19:52:03.475169 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.475122    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-envoy\") pod \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") "
Nov 03 19:52:03.475315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.475225    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-token\" (UniqueName: \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-token\") pod \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") "
Nov 03 19:52:03.475315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.475256    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-certs\") pod \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") "
Nov 03 19:52:03.475315 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.475290    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-data\") pod \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") "
Nov 03 19:52:03.475476 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.475362    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-podinfo\") pod \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") "
Nov 03 19:52:03.475538 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.475505    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-kube-api-access-64sv7\" (UniqueName: \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-kube-api-access-64sv7\") pod \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") "
Nov 03 19:52:03.475598 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.475556    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istiod-ca-cert\") pod \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") "
Nov 03 19:52:03.475658 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.475608    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-socket\") pod \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\" (UID: \"a72d5d6f-7d72-4f4b-aba0-8c884a3723f8\") "
Nov 03 19:52:03.476376 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.476349    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-socket" pod "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" (UID: "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:03.476592 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.476570    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-credential-socket" pod "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" (UID: "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:03.477084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.477024    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-data" pod "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" (UID: "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:03.477253 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.477230    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istiod-ca-cert" pod "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" (UID: "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:03.477806 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.477784    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-certs" pod "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" (UID: "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:03.480501 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.480476    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-envoy" pod "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" (UID: "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:03.480863 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.480840    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-podinfo" pod "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" (UID: "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:52:03.486423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.486393    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-kube-api-access-64sv7" pod "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" (UID: "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8"). InnerVolumeSpecName "kube-api-access-64sv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:03.486525 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.486444    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-token" pod "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" (UID: "a72d5d6f-7d72-4f4b-aba0-8c884a3723f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:03.577057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.577008    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.577057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.577053    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.577057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.577065    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.577325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.577074    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.577325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.577083    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64sv7\" (UniqueName: \"kubernetes.io/projected/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-kube-api-access-64sv7\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.577325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.577093    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.577325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.577102    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.577325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.577110    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.577325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.577118    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.750627 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.750601    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:52:03.807751 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.807718    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"]
Nov 03 19:52:03.817284 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.812407    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-thtmf"]
Nov 03 19:52:03.882593 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.882551    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dab38bfe-c49e-4031-8391-6075964ace09-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab38bfe-c49e-4031-8391-6075964ace09-istio-podinfo\") pod \"dab38bfe-c49e-4031-8391-6075964ace09\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") "
Nov 03 19:52:03.882748 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.882715    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-certs\") pod \"dab38bfe-c49e-4031-8391-6075964ace09\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") "
Nov 03 19:52:03.882795 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.882748    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-kube-api-access-ltl67\" (UniqueName: \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-kube-api-access-ltl67\") pod \"dab38bfe-c49e-4031-8391-6075964ace09\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") "
Nov 03 19:52:03.882830 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.882804    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dab38bfe-c49e-4031-8391-6075964ace09-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab38bfe-c49e-4031-8391-6075964ace09-istiod-ca-cert\") pod \"dab38bfe-c49e-4031-8391-6075964ace09\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") "
Nov 03 19:52:03.882884 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.882856    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-data\") pod \"dab38bfe-c49e-4031-8391-6075964ace09\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") "
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.883098    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-certs" pod "dab38bfe-c49e-4031-8391-6075964ace09" (UID: "dab38bfe-c49e-4031-8391-6075964ace09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.883244    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-data" pod "dab38bfe-c49e-4031-8391-6075964ace09" (UID: "dab38bfe-c49e-4031-8391-6075964ace09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.883371    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dab38bfe-c49e-4031-8391-6075964ace09-istiod-ca-cert" pod "dab38bfe-c49e-4031-8391-6075964ace09" (UID: "dab38bfe-c49e-4031-8391-6075964ace09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.883577    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-istio-token\" (UniqueName: \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-istio-token\") pod \"dab38bfe-c49e-4031-8391-6075964ace09\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") "
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.883621    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-socket\") pod \"dab38bfe-c49e-4031-8391-6075964ace09\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") "
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.883705    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-credential-socket\") pod \"dab38bfe-c49e-4031-8391-6075964ace09\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") "
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.883742    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-envoy\") pod \"dab38bfe-c49e-4031-8391-6075964ace09\" (UID: \"dab38bfe-c49e-4031-8391-6075964ace09\") "
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.884013    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-credential-socket" pod "dab38bfe-c49e-4031-8391-6075964ace09" (UID: "dab38bfe-c49e-4031-8391-6075964ace09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.884398    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.884419    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.884433    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dab38bfe-c49e-4031-8391-6075964ace09-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.884447    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.884719 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.884617    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-socket" pod "dab38bfe-c49e-4031-8391-6075964ace09" (UID: "dab38bfe-c49e-4031-8391-6075964ace09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:03.886316 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.886291    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-kube-api-access-ltl67" pod "dab38bfe-c49e-4031-8391-6075964ace09" (UID: "dab38bfe-c49e-4031-8391-6075964ace09"). InnerVolumeSpecName "kube-api-access-ltl67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:03.887105 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.887080    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dab38bfe-c49e-4031-8391-6075964ace09-istio-podinfo" pod "dab38bfe-c49e-4031-8391-6075964ace09" (UID: "dab38bfe-c49e-4031-8391-6075964ace09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:52:03.887833 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.887804    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-istio-token" pod "dab38bfe-c49e-4031-8391-6075964ace09" (UID: "dab38bfe-c49e-4031-8391-6075964ace09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:03.888570 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.888539    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-envoy" pod "dab38bfe-c49e-4031-8391-6075964ace09" (UID: "dab38bfe-c49e-4031-8391-6075964ace09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:03.986119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.986085    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.986119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.986114    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.986119 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.986125    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dab38bfe-c49e-4031-8391-6075964ace09-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.986354 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.986133    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dab38bfe-c49e-4031-8391-6075964ace09-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:03.986354 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:03.986143    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltl67\" (UniqueName: \"kubernetes.io/projected/dab38bfe-c49e-4031-8391-6075964ace09-kube-api-access-ltl67\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:04.455251 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:04.455166    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"
Nov 03 19:52:04.455251 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:04.455219    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89" event={"ID":"dab38bfe-c49e-4031-8391-6075964ace09","Type":"ContainerDied","Data":"2ba6e16dd1ed649b6264ee9fc8b407407da901ede8f24ab877892a7de2b23dc4"}
Nov 03 19:52:04.455478 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:04.455277    5051 scope.go:122] "RemoveContainer" containerID="72049b7fe571b009faf919b1c3b84d0606c1f6a75526a924aa42173a1a8d558e"
Nov 03 19:52:04.486545 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:04.486516    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"]
Nov 03 19:52:04.494057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:04.494019    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c54f85-openshift-default-5d756c69f5-67l89"]
Nov 03 19:52:05.229693 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:05.229659    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" path="/var/lib/kubelet/pods/a72d5d6f-7d72-4f4b-aba0-8c884a3723f8/volumes"
Nov 03 19:52:05.230228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:05.230212    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dab38bfe-c49e-4031-8391-6075964ace09" path="/var/lib/kubelet/pods/dab38bfe-c49e-4031-8391-6075964ace09/volumes"
Nov 03 19:52:05.466945 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:05.466910    5051 generic.go:356] "Generic (PLEG): container finished" podID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerID="aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e" exitCode=0
Nov 03 19:52:05.467143 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:05.466992    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" event={"ID":"deae5dcc-626c-4635-a8ff-455a48be49e1","Type":"ContainerDied","Data":"aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e"}
Nov 03 19:52:05.467520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:05.467501    5051 scope.go:122] "RemoveContainer" containerID="aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e"
Nov 03 19:52:05.469179 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:05.469151    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:52:05.469278 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:05.469217    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:52:05.469278 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:05.469248    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:52:06.387792 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.387759    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"]
Nov 03 19:52:06.388600 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.388565    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs" podUID="610d96fe-f68f-4876-b85a-de3017180cf5" containerName="maas-api" containerID="cri-o://40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29" gracePeriod=30
Nov 03 19:52:06.478850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.476114    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" event={"ID":"deae5dcc-626c-4635-a8ff-455a48be49e1","Type":"ContainerStarted","Data":"abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2"}
Nov 03 19:52:06.568970 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.568928    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:52:06.569258 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.569234    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 03 19:52:06.569340 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.569285    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 03 19:52:06.811852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.811828    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:52:06.966066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.965692    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-metrics-certs\") pod \"610d96fe-f68f-4876-b85a-de3017180cf5\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") "
Nov 03 19:52:06.966066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.966016    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/610d96fe-f68f-4876-b85a-de3017180cf5-kube-api-access-g24vl\" (UniqueName: \"kubernetes.io/projected/610d96fe-f68f-4876-b85a-de3017180cf5-kube-api-access-g24vl\") pod \"610d96fe-f68f-4876-b85a-de3017180cf5\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") "
Nov 03 19:52:06.966796 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.966506    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-maas-api-tls\") pod \"610d96fe-f68f-4876-b85a-de3017180cf5\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") "
Nov 03 19:52:06.967004 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.966983    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/610d96fe-f68f-4876-b85a-de3017180cf5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/610d96fe-f68f-4876-b85a-de3017180cf5-openshift-service-ca\") pod \"610d96fe-f68f-4876-b85a-de3017180cf5\" (UID: \"610d96fe-f68f-4876-b85a-de3017180cf5\") "
Nov 03 19:52:06.967563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.967534    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/610d96fe-f68f-4876-b85a-de3017180cf5-openshift-service-ca" pod "610d96fe-f68f-4876-b85a-de3017180cf5" (UID: "610d96fe-f68f-4876-b85a-de3017180cf5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:06.971561 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.971531    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/610d96fe-f68f-4876-b85a-de3017180cf5-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:06.973846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.973804    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-maas-api-tls" pod "610d96fe-f68f-4876-b85a-de3017180cf5" (UID: "610d96fe-f68f-4876-b85a-de3017180cf5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:06.976095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.974076    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-metrics-certs" pod "610d96fe-f68f-4876-b85a-de3017180cf5" (UID: "610d96fe-f68f-4876-b85a-de3017180cf5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:06.976095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:06.975406    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/610d96fe-f68f-4876-b85a-de3017180cf5-kube-api-access-g24vl" pod "610d96fe-f68f-4876-b85a-de3017180cf5" (UID: "610d96fe-f68f-4876-b85a-de3017180cf5"). InnerVolumeSpecName "kube-api-access-g24vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:07.074922 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.074520    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:07.074922 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.074548    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g24vl\" (UniqueName: \"kubernetes.io/projected/610d96fe-f68f-4876-b85a-de3017180cf5-kube-api-access-g24vl\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:07.074922 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.074561    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/610d96fe-f68f-4876-b85a-de3017180cf5-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:07.484155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.484120    5051 generic.go:356] "Generic (PLEG): container finished" podID="610d96fe-f68f-4876-b85a-de3017180cf5" containerID="40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29" exitCode=0
Nov 03 19:52:07.484594 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.484176    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"
Nov 03 19:52:07.484594 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.484212    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs" event={"ID":"610d96fe-f68f-4876-b85a-de3017180cf5","Type":"ContainerDied","Data":"40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29"}
Nov 03 19:52:07.484594 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.484258    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs" event={"ID":"610d96fe-f68f-4876-b85a-de3017180cf5","Type":"ContainerDied","Data":"ffb933237b38efb12747ac20cd09100135ef295dfe43ad3838e1f40a7b354008"}
Nov 03 19:52:07.484594 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.484280    5051 scope.go:122] "RemoveContainer" containerID="40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29"
Nov 03 19:52:07.495083 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.495028    5051 scope.go:122] "RemoveContainer" containerID="40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29"
Nov 03 19:52:07.495449 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:07.495426    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29\": container with ID starting with 40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29 not found: ID does not exist" containerID="40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29"
Nov 03 19:52:07.495501 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.495460    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29"} err="failed to get container status \"40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29\": rpc error: code = NotFound desc = could not find container \"40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29\": container with ID starting with 40b680019d6b2f3e8a385c8bce587e4e25b8d49f5786f47e372613208f367e29 not found: ID does not exist"
Nov 03 19:52:07.504339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.504301    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"]
Nov 03 19:52:07.509892 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.509859    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs"]
Nov 03 19:52:07.568828 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.568790    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 03 19:52:07.569061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:07.568864    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 03 19:52:08.568917 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:08.568873    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 03 19:52:08.569440 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:08.568933    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 03 19:52:08.674622 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:08.674583    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"]
Nov 03 19:52:08.674869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:08.674843    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy" containerID="cri-o://01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd" gracePeriod=30
Nov 03 19:52:09.232769 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:09.232735    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="610d96fe-f68f-4876-b85a-de3017180cf5" path="/var/lib/kubelet/pods/610d96fe-f68f-4876-b85a-de3017180cf5/volumes"
Nov 03 19:52:09.568965 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:09.568884    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 03 19:52:09.568965 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:09.568948    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 03 19:52:09.711252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:09.711211    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2c3fc4-84ffb8f66f-4dtcs" podUID="610d96fe-f68f-4876-b85a-de3017180cf5" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.63:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:52:10.569063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:10.568489    5051 patch_prober.go:29] interesting pod/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 03 19:52:10.569063 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:10.568570    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 03 19:52:11.607245 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.607190    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"]
Nov 03 19:52:11.607865 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.607536    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2" podUID="8c49123a-fa09-4756-8083-6d3a5c4d3f48" containerName="maas-api" containerID="cri-o://7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1" gracePeriod=30
Nov 03 19:52:11.711503 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.711474    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:52:11.711791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.711762    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:52:11.712687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.712667    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:52:11.869324 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.869245    5051 patch_prober.go:29] interesting pod/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 19:52:11.869465 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.869317    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 19:52:11.931873 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.931848    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:52:11.944444 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.944416    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"]
Nov 03 19:52:11.976609 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.976578    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c49123a-fa09-4756-8083-6d3a5c4d3f48-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c49123a-fa09-4756-8083-6d3a5c4d3f48-openshift-service-ca\") pod \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") "
Nov 03 19:52:11.976800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.976620    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-metrics-certs\") pod \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") "
Nov 03 19:52:11.976800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.976694    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-maas-api-tls\") pod \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") "
Nov 03 19:52:11.976937 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.976801    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c49123a-fa09-4756-8083-6d3a5c4d3f48-kube-api-access-s447d\" (UniqueName: \"kubernetes.io/projected/8c49123a-fa09-4756-8083-6d3a5c4d3f48-kube-api-access-s447d\") pod \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\" (UID: \"8c49123a-fa09-4756-8083-6d3a5c4d3f48\") "
Nov 03 19:52:11.977090 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.977032    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c49123a-fa09-4756-8083-6d3a5c4d3f48-openshift-service-ca" pod "8c49123a-fa09-4756-8083-6d3a5c4d3f48" (UID: "8c49123a-fa09-4756-8083-6d3a5c4d3f48"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:11.977478 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.977453    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c49123a-fa09-4756-8083-6d3a5c4d3f48-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:11.979023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.978997    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-metrics-certs" pod "8c49123a-fa09-4756-8083-6d3a5c4d3f48" (UID: "8c49123a-fa09-4756-8083-6d3a5c4d3f48"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:11.979505 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.979482    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c49123a-fa09-4756-8083-6d3a5c4d3f48-kube-api-access-s447d" pod "8c49123a-fa09-4756-8083-6d3a5c4d3f48" (UID: "8c49123a-fa09-4756-8083-6d3a5c4d3f48"). InnerVolumeSpecName "kube-api-access-s447d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:11.980263 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:11.979976    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-maas-api-tls" pod "8c49123a-fa09-4756-8083-6d3a5c4d3f48" (UID: "8c49123a-fa09-4756-8083-6d3a5c4d3f48"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:12.078704 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.078672    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:12.078704 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.078702    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c49123a-fa09-4756-8083-6d3a5c4d3f48-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:12.078704 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.078712    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s447d\" (UniqueName: \"kubernetes.io/projected/8c49123a-fa09-4756-8083-6d3a5c4d3f48-kube-api-access-s447d\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:12.519659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.519623    5051 generic.go:356] "Generic (PLEG): container finished" podID="8c49123a-fa09-4756-8083-6d3a5c4d3f48" containerID="7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1" exitCode=0
Nov 03 19:52:12.519659 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.519660    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"
Nov 03 19:52:12.519873 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.519705    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2" event={"ID":"8c49123a-fa09-4756-8083-6d3a5c4d3f48","Type":"ContainerDied","Data":"7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1"}
Nov 03 19:52:12.519873 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.519766    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2" event={"ID":"8c49123a-fa09-4756-8083-6d3a5c4d3f48","Type":"ContainerDied","Data":"ad11f5851d47409a9bab04bac3c505995e723c4a5feb13ae4c7229344d00c0bc"}
Nov 03 19:52:12.519873 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.519797    5051 scope.go:122] "RemoveContainer" containerID="7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1"
Nov 03 19:52:12.520503 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.520462    5051 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" secret="" err="secret \"e2e-shared-a-w2-29ab98-openshift-default-dockercfg-x7k8r\" not found"
Nov 03 19:52:12.533415 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.533390    5051 scope.go:122] "RemoveContainer" containerID="7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1"
Nov 03 19:52:12.533787 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:12.533764    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1\": container with ID starting with 7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1 not found: ID does not exist" containerID="7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1"
Nov 03 19:52:12.533875 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.533799    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1"} err="failed to get container status \"7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1\": rpc error: code = NotFound desc = could not find container \"7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1\": container with ID starting with 7fba7cf8725bd397679008126a5988a1edcd3503bd788653c8efd3640c282ee1 not found: ID does not exist"
Nov 03 19:52:12.550057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.547371    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"]
Nov 03 19:52:12.550376 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.550350    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab584a-598d98656c-dp9r2"]
Nov 03 19:52:12.711486 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:12.711447    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:52:13.229720 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:13.229688    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c49123a-fa09-4756-8083-6d3a5c4d3f48" path="/var/lib/kubelet/pods/8c49123a-fa09-4756-8083-6d3a5c4d3f48/volumes"
Nov 03 19:52:13.526902 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:13.526805    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy" containerID="cri-o://abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2" gracePeriod=30
Nov 03 19:52:13.948201 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:13.948177    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:52:14.004235 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.004198    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-istio-token\" (UniqueName: \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-istio-token\") pod \"d7655d6f-386a-4364-9343-6e56877d3faa\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") "
Nov 03 19:52:14.004395 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.004249    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-data\") pod \"d7655d6f-386a-4364-9343-6e56877d3faa\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") "
Nov 03 19:52:14.004395 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.004336    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-certs\") pod \"d7655d6f-386a-4364-9343-6e56877d3faa\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") "
Nov 03 19:52:14.004395 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.004376    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7655d6f-386a-4364-9343-6e56877d3faa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7655d6f-386a-4364-9343-6e56877d3faa-istiod-ca-cert\") pod \"d7655d6f-386a-4364-9343-6e56877d3faa\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") "
Nov 03 19:52:14.004586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.004464    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-kube-api-access-fdgzl\" (UniqueName: \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-kube-api-access-fdgzl\") pod \"d7655d6f-386a-4364-9343-6e56877d3faa\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") "
Nov 03 19:52:14.004586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.004554    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-credential-socket\") pod \"d7655d6f-386a-4364-9343-6e56877d3faa\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") "
Nov 03 19:52:14.004708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.004589    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-socket\") pod \"d7655d6f-386a-4364-9343-6e56877d3faa\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") "
Nov 03 19:52:14.004708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.004617    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-envoy\") pod \"d7655d6f-386a-4364-9343-6e56877d3faa\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") "
Nov 03 19:52:14.004708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.004624    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-data" pod "d7655d6f-386a-4364-9343-6e56877d3faa" (UID: "d7655d6f-386a-4364-9343-6e56877d3faa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:14.004708 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.004683    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-certs" pod "d7655d6f-386a-4364-9343-6e56877d3faa" (UID: "d7655d6f-386a-4364-9343-6e56877d3faa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:14.004934 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.004719    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d7655d6f-386a-4364-9343-6e56877d3faa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7655d6f-386a-4364-9343-6e56877d3faa-istio-podinfo\") pod \"d7655d6f-386a-4364-9343-6e56877d3faa\" (UID: \"d7655d6f-386a-4364-9343-6e56877d3faa\") "
Nov 03 19:52:14.005160 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.005098    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-credential-socket" pod "d7655d6f-386a-4364-9343-6e56877d3faa" (UID: "d7655d6f-386a-4364-9343-6e56877d3faa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:14.005335 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.005302    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-socket" pod "d7655d6f-386a-4364-9343-6e56877d3faa" (UID: "d7655d6f-386a-4364-9343-6e56877d3faa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:14.005755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.005503    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:14.005755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.005529    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:14.005755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.005546    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:14.005755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.005560    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:14.005755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.005725    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7655d6f-386a-4364-9343-6e56877d3faa-istiod-ca-cert" pod "d7655d6f-386a-4364-9343-6e56877d3faa" (UID: "d7655d6f-386a-4364-9343-6e56877d3faa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:14.006601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.006571    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-istio-token" pod "d7655d6f-386a-4364-9343-6e56877d3faa" (UID: "d7655d6f-386a-4364-9343-6e56877d3faa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:14.007156 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.007138    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-envoy" pod "d7655d6f-386a-4364-9343-6e56877d3faa" (UID: "d7655d6f-386a-4364-9343-6e56877d3faa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:14.007276 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.007261    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-kube-api-access-fdgzl" pod "d7655d6f-386a-4364-9343-6e56877d3faa" (UID: "d7655d6f-386a-4364-9343-6e56877d3faa"). InnerVolumeSpecName "kube-api-access-fdgzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:14.007411 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.007391    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7655d6f-386a-4364-9343-6e56877d3faa-istio-podinfo" pod "d7655d6f-386a-4364-9343-6e56877d3faa" (UID: "d7655d6f-386a-4364-9343-6e56877d3faa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:52:14.106714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.106630    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdgzl\" (UniqueName: \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-kube-api-access-fdgzl\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:14.106714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.106661    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7655d6f-386a-4364-9343-6e56877d3faa-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:14.106714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.106675    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7655d6f-386a-4364-9343-6e56877d3faa-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:14.106714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.106687    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7655d6f-386a-4364-9343-6e56877d3faa-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:14.106714 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.106700    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7655d6f-386a-4364-9343-6e56877d3faa-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:14.534841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.534810    5051 generic.go:356] "Generic (PLEG): container finished" podID="d7655d6f-386a-4364-9343-6e56877d3faa" containerID="01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd" exitCode=0
Nov 03 19:52:14.535022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.534868    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"
Nov 03 19:52:14.535022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.534899    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" event={"ID":"d7655d6f-386a-4364-9343-6e56877d3faa","Type":"ContainerDied","Data":"01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd"}
Nov 03 19:52:14.535022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.534958    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2" event={"ID":"d7655d6f-386a-4364-9343-6e56877d3faa","Type":"ContainerDied","Data":"b7c48c89210cdc178e05dfaaacd6354211504a214c59ecdb95a2013e871c53e8"}
Nov 03 19:52:14.535022 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.534988    5051 scope.go:122] "RemoveContainer" containerID="01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd"
Nov 03 19:52:14.546215 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.546196    5051 scope.go:122] "RemoveContainer" containerID="219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2"
Nov 03 19:52:14.555726 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.555702    5051 scope.go:122] "RemoveContainer" containerID="01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd"
Nov 03 19:52:14.555983 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:14.555962    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd\": container with ID starting with 01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd not found: ID does not exist" containerID="01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd"
Nov 03 19:52:14.556078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.555996    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd"} err="failed to get container status \"01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd\": rpc error: code = NotFound desc = could not find container \"01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd\": container with ID starting with 01c643cd3d756fccf0ed4329a2c8d6099722c3a6857c890e08840ffea4b3acdd not found: ID does not exist"
Nov 03 19:52:14.556078 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.556020    5051 scope.go:122] "RemoveContainer" containerID="219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2"
Nov 03 19:52:14.556294 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:14.556276    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2\": container with ID starting with 219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2 not found: ID does not exist" containerID="219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2"
Nov 03 19:52:14.556342 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.556299    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2"} err="failed to get container status \"219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2\": rpc error: code = NotFound desc = could not find container \"219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2\": container with ID starting with 219cd34c13e2238361a83a7ddce55a226214a86ac902281aae17cda0604dffc2 not found: ID does not exist"
Nov 03 19:52:14.587492 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.587468    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"]
Nov 03 19:52:14.595755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:14.595733    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-9ce34d-openshift-default-7fc4849fd5-2bvw2"]
Nov 03 19:52:15.229542 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:15.229498    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" path="/var/lib/kubelet/pods/d7655d6f-386a-4364-9343-6e56877d3faa/volumes"
Nov 03 19:52:18.787500 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.787445    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:52:18.868672 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.868642    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-podinfo\") pod \"deae5dcc-626c-4635-a8ff-455a48be49e1\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") "
Nov 03 19:52:18.868845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.868826    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-certs\") pod \"deae5dcc-626c-4635-a8ff-455a48be49e1\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") "
Nov 03 19:52:18.868913 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.868904    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-token\" (UniqueName: \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-token\") pod \"deae5dcc-626c-4635-a8ff-455a48be49e1\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") "
Nov 03 19:52:18.869058 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869024    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-socket\") pod \"deae5dcc-626c-4635-a8ff-455a48be49e1\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") "
Nov 03 19:52:18.869144 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869090    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-envoy\") pod \"deae5dcc-626c-4635-a8ff-455a48be49e1\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") "
Nov 03 19:52:18.869144 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869133    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/deae5dcc-626c-4635-a8ff-455a48be49e1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deae5dcc-626c-4635-a8ff-455a48be49e1-istiod-ca-cert\") pod \"deae5dcc-626c-4635-a8ff-455a48be49e1\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") "
Nov 03 19:52:18.869263 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869202    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-data\") pod \"deae5dcc-626c-4635-a8ff-455a48be49e1\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") "
Nov 03 19:52:18.869263 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869187    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-certs" pod "deae5dcc-626c-4635-a8ff-455a48be49e1" (UID: "deae5dcc-626c-4635-a8ff-455a48be49e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:18.869368 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869269    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-kube-api-access-rtf2f\" (UniqueName: \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-kube-api-access-rtf2f\") pod \"deae5dcc-626c-4635-a8ff-455a48be49e1\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") "
Nov 03 19:52:18.869368 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869307    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-credential-socket\") pod \"deae5dcc-626c-4635-a8ff-455a48be49e1\" (UID: \"deae5dcc-626c-4635-a8ff-455a48be49e1\") "
Nov 03 19:52:18.869613 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869530    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deae5dcc-626c-4635-a8ff-455a48be49e1-istiod-ca-cert" pod "deae5dcc-626c-4635-a8ff-455a48be49e1" (UID: "deae5dcc-626c-4635-a8ff-455a48be49e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:18.869827 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869810    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:18.869898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869833    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deae5dcc-626c-4635-a8ff-455a48be49e1-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:18.869898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869826    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-data" pod "deae5dcc-626c-4635-a8ff-455a48be49e1" (UID: "deae5dcc-626c-4635-a8ff-455a48be49e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:18.869898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869842    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-credential-socket" pod "deae5dcc-626c-4635-a8ff-455a48be49e1" (UID: "deae5dcc-626c-4635-a8ff-455a48be49e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:18.870025 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.869923    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-socket" pod "deae5dcc-626c-4635-a8ff-455a48be49e1" (UID: "deae5dcc-626c-4635-a8ff-455a48be49e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:18.871150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.871125    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-podinfo" pod "deae5dcc-626c-4635-a8ff-455a48be49e1" (UID: "deae5dcc-626c-4635-a8ff-455a48be49e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:52:18.871530 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.871504    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-token" pod "deae5dcc-626c-4635-a8ff-455a48be49e1" (UID: "deae5dcc-626c-4635-a8ff-455a48be49e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:18.871829 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.871805    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-kube-api-access-rtf2f" pod "deae5dcc-626c-4635-a8ff-455a48be49e1" (UID: "deae5dcc-626c-4635-a8ff-455a48be49e1"). InnerVolumeSpecName "kube-api-access-rtf2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:18.872087 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.872017    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-envoy" pod "deae5dcc-626c-4635-a8ff-455a48be49e1" (UID: "deae5dcc-626c-4635-a8ff-455a48be49e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:18.970645 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.970620    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:18.970645 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.970645    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:18.970784 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.970654    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:18.970784 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.970668    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:18.970784 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.970679    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtf2f\" (UniqueName: \"kubernetes.io/projected/deae5dcc-626c-4635-a8ff-455a48be49e1-kube-api-access-rtf2f\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:18.970784 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.970688    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deae5dcc-626c-4635-a8ff-455a48be49e1-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:18.970784 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:18.970698    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deae5dcc-626c-4635-a8ff-455a48be49e1-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:19.584751 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.584659    5051 generic.go:356] "Generic (PLEG): container finished" podID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerID="abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2" exitCode=0
Nov 03 19:52:19.584751 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.584741    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"
Nov 03 19:52:19.584937 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.584769    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" event={"ID":"deae5dcc-626c-4635-a8ff-455a48be49e1","Type":"ContainerDied","Data":"abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2"}
Nov 03 19:52:19.584937 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.584834    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z" event={"ID":"deae5dcc-626c-4635-a8ff-455a48be49e1","Type":"ContainerDied","Data":"271b88c550ef23dd54c58793dbb2fc04b07052216a5b92c441a28ee60e7710f7"}
Nov 03 19:52:19.584937 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.584866    5051 scope.go:122] "RemoveContainer" containerID="abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2"
Nov 03 19:52:19.598874 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.598831    5051 scope.go:122] "RemoveContainer" containerID="aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e"
Nov 03 19:52:19.608349 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.608324    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"]
Nov 03 19:52:19.612599 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.612568    5051 scope.go:122] "RemoveContainer" containerID="abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2"
Nov 03 19:52:19.612863 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:19.612842    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2\": container with ID starting with abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2 not found: ID does not exist" containerID="abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2"
Nov 03 19:52:19.612942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.612872    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2"} err="failed to get container status \"abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2\": rpc error: code = NotFound desc = could not find container \"abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2\": container with ID starting with abb437ea7105078e0f888df493fb1348f97049ae068ef301f4e81243f258a6c2 not found: ID does not exist"
Nov 03 19:52:19.612942 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.612889    5051 scope.go:122] "RemoveContainer" containerID="aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e"
Nov 03 19:52:19.613235 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:19.613217    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e\": container with ID starting with aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e not found: ID does not exist" containerID="aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e"
Nov 03 19:52:19.613296 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.613241    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e"} err="failed to get container status \"aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e\": rpc error: code = NotFound desc = could not find container \"aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e\": container with ID starting with aeffbd6d8992649df115257d621529368c7c12529d05e89718550235402c8a2e not found: ID does not exist"
Nov 03 19:52:19.613374 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:19.613352    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-29ab98-openshift-default-67c5b6986f-pfx9z"]
Nov 03 19:52:21.228909 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:21.228874    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" path="/var/lib/kubelet/pods/deae5dcc-626c-4635-a8ff-455a48be49e1/volumes"
Nov 03 19:52:21.396235 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:21.396197    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"]
Nov 03 19:52:21.396482 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:21.396450    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy" containerID="cri-o://839aec472d8840e31cc8b15ee4ba27c32d084af4c6c06577cfded2ab02a10348" gracePeriod=30
Nov 03 19:52:24.999676 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:24.999632    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"]
Nov 03 19:52:25.000188 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:24.999862    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g" podUID="8311401f-491c-43a6-a854-d3ae1f3255ae" containerName="maas-api" containerID="cri-o://bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264" gracePeriod=30
Nov 03 19:52:25.021470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.021435    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g" podUID="8311401f-491c-43a6-a854-d3ae1f3255ae" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.115:8443/health\": dial tcp 10.132.0.115:8443: connect: connection refused"
Nov 03 19:52:25.260761 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.260703    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:52:25.355157 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.355129    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8311401f-491c-43a6-a854-d3ae1f3255ae-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8311401f-491c-43a6-a854-d3ae1f3255ae-openshift-service-ca\") pod \"8311401f-491c-43a6-a854-d3ae1f3255ae\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") "
Nov 03 19:52:25.355330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.355168    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8311401f-491c-43a6-a854-d3ae1f3255ae-kube-api-access-x94ft\" (UniqueName: \"kubernetes.io/projected/8311401f-491c-43a6-a854-d3ae1f3255ae-kube-api-access-x94ft\") pod \"8311401f-491c-43a6-a854-d3ae1f3255ae\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") "
Nov 03 19:52:25.355330 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.355253    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-metrics-certs\") pod \"8311401f-491c-43a6-a854-d3ae1f3255ae\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") "
Nov 03 19:52:25.355459 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.355360    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-maas-api-tls\") pod \"8311401f-491c-43a6-a854-d3ae1f3255ae\" (UID: \"8311401f-491c-43a6-a854-d3ae1f3255ae\") "
Nov 03 19:52:25.355562 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.355539    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8311401f-491c-43a6-a854-d3ae1f3255ae-openshift-service-ca" pod "8311401f-491c-43a6-a854-d3ae1f3255ae" (UID: "8311401f-491c-43a6-a854-d3ae1f3255ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:25.355890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.355868    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8311401f-491c-43a6-a854-d3ae1f3255ae-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:25.357462 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.357440    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8311401f-491c-43a6-a854-d3ae1f3255ae-kube-api-access-x94ft" pod "8311401f-491c-43a6-a854-d3ae1f3255ae" (UID: "8311401f-491c-43a6-a854-d3ae1f3255ae"). InnerVolumeSpecName "kube-api-access-x94ft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:25.357858 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.357830    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-metrics-certs" pod "8311401f-491c-43a6-a854-d3ae1f3255ae" (UID: "8311401f-491c-43a6-a854-d3ae1f3255ae"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:25.357935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.357877    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-maas-api-tls" pod "8311401f-491c-43a6-a854-d3ae1f3255ae" (UID: "8311401f-491c-43a6-a854-d3ae1f3255ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:25.464067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.456972    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x94ft\" (UniqueName: \"kubernetes.io/projected/8311401f-491c-43a6-a854-d3ae1f3255ae-kube-api-access-x94ft\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:25.464067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.457014    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:25.464067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.457031    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8311401f-491c-43a6-a854-d3ae1f3255ae-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:25.635194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.635106    5051 generic.go:356] "Generic (PLEG): container finished" podID="8311401f-491c-43a6-a854-d3ae1f3255ae" containerID="bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264" exitCode=0
Nov 03 19:52:25.635456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.635236    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g" event={"ID":"8311401f-491c-43a6-a854-d3ae1f3255ae","Type":"ContainerDied","Data":"bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264"}
Nov 03 19:52:25.635456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.635285    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g" event={"ID":"8311401f-491c-43a6-a854-d3ae1f3255ae","Type":"ContainerDied","Data":"6c753c8e104f41983e0cd0753bff14053345feaaa91a3584f6ea4f282116abfe"}
Nov 03 19:52:25.635456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.635313    5051 scope.go:122] "RemoveContainer" containerID="bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264"
Nov 03 19:52:25.635609 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.635472    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"
Nov 03 19:52:25.647140 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.647115    5051 scope.go:122] "RemoveContainer" containerID="bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264"
Nov 03 19:52:25.647412 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:25.647388    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264\": container with ID starting with bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264 not found: ID does not exist" containerID="bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264"
Nov 03 19:52:25.647494 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.647422    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264"} err="failed to get container status \"bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264\": rpc error: code = NotFound desc = could not find container \"bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264\": container with ID starting with bd4c934a85a68dbfb9acdb844acbb6fe933e0a1f7e886dd1134fc139f8f6c264 not found: ID does not exist"
Nov 03 19:52:25.662118 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.662092    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"]
Nov 03 19:52:25.665712 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:25.665689    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-25944c-665877b594-bfd2g"]
Nov 03 19:52:26.644893 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.644866    5051 generic.go:356] "Generic (PLEG): container finished" podID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerID="839aec472d8840e31cc8b15ee4ba27c32d084af4c6c06577cfded2ab02a10348" exitCode=0
Nov 03 19:52:26.645312 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.644943    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" event={"ID":"4761a958-3fed-4c9f-aa46-314065cebb4a","Type":"ContainerDied","Data":"839aec472d8840e31cc8b15ee4ba27c32d084af4c6c06577cfded2ab02a10348"}
Nov 03 19:52:26.645312 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.645021    5051 scope.go:122] "RemoveContainer" containerID="a1e47d98d2349afc93b60d10631ea08a066de639c970ee387ce5c2f459482c00"
Nov 03 19:52:26.669184 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.669164    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:52:26.770805 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.770772    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-envoy\") pod \"4761a958-3fed-4c9f-aa46-314065cebb4a\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") "
Nov 03 19:52:26.771010 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.770829    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-credential-socket\") pod \"4761a958-3fed-4c9f-aa46-314065cebb4a\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") "
Nov 03 19:52:26.771010 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.770890    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-socket\") pod \"4761a958-3fed-4c9f-aa46-314065cebb4a\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") "
Nov 03 19:52:26.771010 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.770961    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-data\") pod \"4761a958-3fed-4c9f-aa46-314065cebb4a\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") "
Nov 03 19:52:26.771223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.771011    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-podinfo\") pod \"4761a958-3fed-4c9f-aa46-314065cebb4a\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") "
Nov 03 19:52:26.771223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.771063    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-token\" (UniqueName: \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-token\") pod \"4761a958-3fed-4c9f-aa46-314065cebb4a\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") "
Nov 03 19:52:26.771223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.771117    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-kube-api-access-jsfwq\" (UniqueName: \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-kube-api-access-jsfwq\") pod \"4761a958-3fed-4c9f-aa46-314065cebb4a\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") "
Nov 03 19:52:26.771223 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.771211    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-socket" pod "4761a958-3fed-4c9f-aa46-314065cebb4a" (UID: "4761a958-3fed-4c9f-aa46-314065cebb4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:26.771497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.771229    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-credential-socket" pod "4761a958-3fed-4c9f-aa46-314065cebb4a" (UID: "4761a958-3fed-4c9f-aa46-314065cebb4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:26.771497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.771264    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-data" pod "4761a958-3fed-4c9f-aa46-314065cebb4a" (UID: "4761a958-3fed-4c9f-aa46-314065cebb4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:26.771497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.771266    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4761a958-3fed-4c9f-aa46-314065cebb4a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4761a958-3fed-4c9f-aa46-314065cebb4a-istiod-ca-cert\") pod \"4761a958-3fed-4c9f-aa46-314065cebb4a\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") "
Nov 03 19:52:26.771497 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.771372    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-certs\") pod \"4761a958-3fed-4c9f-aa46-314065cebb4a\" (UID: \"4761a958-3fed-4c9f-aa46-314065cebb4a\") "
Nov 03 19:52:26.771736 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.771686    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-certs" pod "4761a958-3fed-4c9f-aa46-314065cebb4a" (UID: "4761a958-3fed-4c9f-aa46-314065cebb4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:26.771796 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.771728    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4761a958-3fed-4c9f-aa46-314065cebb4a-istiod-ca-cert" pod "4761a958-3fed-4c9f-aa46-314065cebb4a" (UID: "4761a958-3fed-4c9f-aa46-314065cebb4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:26.772110 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.772088    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4761a958-3fed-4c9f-aa46-314065cebb4a-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:26.772110 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.772111    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:26.772267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.772124    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:26.772267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.772137    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:26.772267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.772150    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:26.773601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.773576    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-podinfo" pod "4761a958-3fed-4c9f-aa46-314065cebb4a" (UID: "4761a958-3fed-4c9f-aa46-314065cebb4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:52:26.773601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.773588    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-token" pod "4761a958-3fed-4c9f-aa46-314065cebb4a" (UID: "4761a958-3fed-4c9f-aa46-314065cebb4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:26.773922 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.773902    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-envoy" pod "4761a958-3fed-4c9f-aa46-314065cebb4a" (UID: "4761a958-3fed-4c9f-aa46-314065cebb4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:26.773993 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.773982    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-kube-api-access-jsfwq" pod "4761a958-3fed-4c9f-aa46-314065cebb4a" (UID: "4761a958-3fed-4c9f-aa46-314065cebb4a"). InnerVolumeSpecName "kube-api-access-jsfwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:26.872953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.872921    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:26.872953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.872948    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsfwq\" (UniqueName: \"kubernetes.io/projected/4761a958-3fed-4c9f-aa46-314065cebb4a-kube-api-access-jsfwq\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:26.872953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.872958    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:26.873207 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.872969    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4761a958-3fed-4c9f-aa46-314065cebb4a-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:26.930419 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.930348    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"]
Nov 03 19:52:26.949797 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:26.949767    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"]
Nov 03 19:52:27.229615 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:27.229582    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8311401f-491c-43a6-a854-d3ae1f3255ae" path="/var/lib/kubelet/pods/8311401f-491c-43a6-a854-d3ae1f3255ae/volumes"
Nov 03 19:52:27.652554 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:27.652476    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"
Nov 03 19:52:27.652554 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:27.652514    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw" event={"ID":"4761a958-3fed-4c9f-aa46-314065cebb4a","Type":"ContainerDied","Data":"63ea5dde67aa3bde02c8789047d08cc2324efddf1029f753f81b3ef10f4c1f03"}
Nov 03 19:52:27.653013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:27.652572    5051 scope.go:122] "RemoveContainer" containerID="839aec472d8840e31cc8b15ee4ba27c32d084af4c6c06577cfded2ab02a10348"
Nov 03 19:52:27.697329 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:27.697300    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"]
Nov 03 19:52:27.705970 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:27.705945    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ab584a-openshift-default-8674b64cd9-tx7lw"]
Nov 03 19:52:29.238532 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:29.238501    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" path="/var/lib/kubelet/pods/4761a958-3fed-4c9f-aa46-314065cebb4a/volumes"
Nov 03 19:52:30.695756 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:30.695722    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"]
Nov 03 19:52:30.696211 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:30.696002    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q" podUID="b740565b-db74-4f04-b916-0b62e4d14ea0" containerName="maas-api" containerID="cri-o://d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07" gracePeriod=30
Nov 03 19:52:30.980800 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:30.980774    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:52:31.021269 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.021234    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-maas-api-tls\") pod \"b740565b-db74-4f04-b916-0b62e4d14ea0\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") "
Nov 03 19:52:31.021462 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.021441    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-metrics-certs\") pod \"b740565b-db74-4f04-b916-0b62e4d14ea0\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") "
Nov 03 19:52:31.021554 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.021521    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b740565b-db74-4f04-b916-0b62e4d14ea0-kube-api-access-nz7dq\" (UniqueName: \"kubernetes.io/projected/b740565b-db74-4f04-b916-0b62e4d14ea0-kube-api-access-nz7dq\") pod \"b740565b-db74-4f04-b916-0b62e4d14ea0\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") "
Nov 03 19:52:31.021621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.021572    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b740565b-db74-4f04-b916-0b62e4d14ea0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b740565b-db74-4f04-b916-0b62e4d14ea0-openshift-service-ca\") pod \"b740565b-db74-4f04-b916-0b62e4d14ea0\" (UID: \"b740565b-db74-4f04-b916-0b62e4d14ea0\") "
Nov 03 19:52:31.022472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.022442    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b740565b-db74-4f04-b916-0b62e4d14ea0-openshift-service-ca" pod "b740565b-db74-4f04-b916-0b62e4d14ea0" (UID: "b740565b-db74-4f04-b916-0b62e4d14ea0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:31.024184 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.024156    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-metrics-certs" pod "b740565b-db74-4f04-b916-0b62e4d14ea0" (UID: "b740565b-db74-4f04-b916-0b62e4d14ea0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:31.026010 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.025973    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-maas-api-tls" pod "b740565b-db74-4f04-b916-0b62e4d14ea0" (UID: "b740565b-db74-4f04-b916-0b62e4d14ea0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:31.026294 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.026257    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b740565b-db74-4f04-b916-0b62e4d14ea0-kube-api-access-nz7dq" pod "b740565b-db74-4f04-b916-0b62e4d14ea0" (UID: "b740565b-db74-4f04-b916-0b62e4d14ea0"). InnerVolumeSpecName "kube-api-access-nz7dq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:31.123015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.122985    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:31.123015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.123013    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz7dq\" (UniqueName: \"kubernetes.io/projected/b740565b-db74-4f04-b916-0b62e4d14ea0-kube-api-access-nz7dq\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:31.123015 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.123024    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b740565b-db74-4f04-b916-0b62e4d14ea0-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:31.123249 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.123033    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b740565b-db74-4f04-b916-0b62e4d14ea0-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:31.680646 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.680551    5051 generic.go:356] "Generic (PLEG): container finished" podID="b740565b-db74-4f04-b916-0b62e4d14ea0" containerID="d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07" exitCode=0
Nov 03 19:52:31.680646 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.680613    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"
Nov 03 19:52:31.680646 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.680628    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q" event={"ID":"b740565b-db74-4f04-b916-0b62e4d14ea0","Type":"ContainerDied","Data":"d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07"}
Nov 03 19:52:31.680891 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.680712    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q" event={"ID":"b740565b-db74-4f04-b916-0b62e4d14ea0","Type":"ContainerDied","Data":"3ff77012ea8c8f8a5d2149828d4555b2c54e79d08e14211125ace0c73f55b4a9"}
Nov 03 19:52:31.680891 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.680736    5051 scope.go:122] "RemoveContainer" containerID="d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07"
Nov 03 19:52:31.691135 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.691113    5051 scope.go:122] "RemoveContainer" containerID="d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07"
Nov 03 19:52:31.691422 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:31.691403    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07\": container with ID starting with d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07 not found: ID does not exist" containerID="d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07"
Nov 03 19:52:31.691498 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.691433    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07"} err="failed to get container status \"d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07\": rpc error: code = NotFound desc = could not find container \"d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07\": container with ID starting with d28d9f57e7914f3a6ef02a54f84a1b1136a17459f3d4460d4eb3ff98abe63a07 not found: ID does not exist"
Nov 03 19:52:31.699928 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.699902    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"]
Nov 03 19:52:31.704860 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:31.704838    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c6a08e-6dbc6949df-9hr2q"]
Nov 03 19:52:32.768796 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:32.768756    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:52:32.804334 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:32.804302    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:52:33.165847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:33.165764    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"]
Nov 03 19:52:33.166257 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:33.166203    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy" containerID="cri-o://19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782" gracePeriod=30
Nov 03 19:52:33.229532 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:33.229496    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b740565b-db74-4f04-b916-0b62e4d14ea0" path="/var/lib/kubelet/pods/b740565b-db74-4f04-b916-0b62e4d14ea0/volumes"
Nov 03 19:52:35.122601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.122559    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"]
Nov 03 19:52:35.123138 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123122    5051 cpu_manager.go:414] "containerMap: removing container" podUID="526cdbab-900d-4990-9a52-deb0b1277d9a" containerName="maas-api"
Nov 03 19:52:35.123200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123140    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="526cdbab-900d-4990-9a52-deb0b1277d9a" containerName="maas-api"
Nov 03 19:52:35.123200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123154    5051 cpu_manager.go:414] "containerMap: removing container" podUID="abcde2a0-cda0-4635-89f3-b791dee9537e" containerName="maas-api"
Nov 03 19:52:35.123200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123160    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abcde2a0-cda0-4635-89f3-b791dee9537e" containerName="maas-api"
Nov 03 19:52:35.123200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123166    5051 cpu_manager.go:414] "containerMap: removing container" podUID="0e1ad4ab-300a-425c-b752-bd6dd6108c4a" containerName="payload-pre-processing"
Nov 03 19:52:35.123200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123171    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e1ad4ab-300a-425c-b752-bd6dd6108c4a" containerName="payload-pre-processing"
Nov 03 19:52:35.123200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123181    5051 cpu_manager.go:414] "containerMap: removing container" podUID="ff0336ae-47dd-45db-9a74-1aaf1bbcb47d" containerName="payload-processing"
Nov 03 19:52:35.123200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123186    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff0336ae-47dd-45db-9a74-1aaf1bbcb47d" containerName="payload-processing"
Nov 03 19:52:35.123200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123195    5051 cpu_manager.go:414] "containerMap: removing container" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy"
Nov 03 19:52:35.123200 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123200    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123206    5051 cpu_manager.go:414] "containerMap: removing container" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123210    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123216    5051 cpu_manager.go:414] "containerMap: removing container" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123221    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123228    5051 cpu_manager.go:414] "containerMap: removing container" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123232    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123238    5051 cpu_manager.go:414] "containerMap: removing container" podUID="3843274d-f1ec-4da5-806f-e841042b92f6" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123242    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3843274d-f1ec-4da5-806f-e841042b92f6" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123247    5051 cpu_manager.go:414] "containerMap: removing container" podUID="dab38bfe-c49e-4031-8391-6075964ace09" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123252    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dab38bfe-c49e-4031-8391-6075964ace09" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123259    5051 cpu_manager.go:414] "containerMap: removing container" podUID="98ab7b99-b947-4052-81f6-dfe33dd18191" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123265    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98ab7b99-b947-4052-81f6-dfe33dd18191" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123272    5051 cpu_manager.go:414] "containerMap: removing container" podUID="610d96fe-f68f-4876-b85a-de3017180cf5" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123277    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="610d96fe-f68f-4876-b85a-de3017180cf5" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123284    5051 cpu_manager.go:414] "containerMap: removing container" podUID="1cf13e8c-1976-4218-943b-4deaf2dde7eb" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123289    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cf13e8c-1976-4218-943b-4deaf2dde7eb" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123299    5051 cpu_manager.go:414] "containerMap: removing container" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123303    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123312    5051 cpu_manager.go:414] "containerMap: removing container" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123316    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123323    5051 cpu_manager.go:414] "containerMap: removing container" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123327    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123337    5051 cpu_manager.go:414] "containerMap: removing container" podUID="9a1a4117-0b43-4d57-8023-25bed714d499" containerName="payload-pre-processing"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123341    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a1a4117-0b43-4d57-8023-25bed714d499" containerName="payload-pre-processing"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123348    5051 cpu_manager.go:414] "containerMap: removing container" podUID="8c49123a-fa09-4756-8083-6d3a5c4d3f48" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123352    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c49123a-fa09-4756-8083-6d3a5c4d3f48" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123361    5051 cpu_manager.go:414] "containerMap: removing container" podUID="82b74e21-57a7-489f-abe4-9814cf683aab" containerName="payload-pre-processing"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123365    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82b74e21-57a7-489f-abe4-9814cf683aab" containerName="payload-pre-processing"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123372    5051 cpu_manager.go:414] "containerMap: removing container" podUID="965226d2-5ae1-4bdf-97b3-6726e9c035d7" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123376    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="965226d2-5ae1-4bdf-97b3-6726e9c035d7" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123386    5051 cpu_manager.go:414] "containerMap: removing container" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123390    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123397    5051 cpu_manager.go:414] "containerMap: removing container" podUID="b740565b-db74-4f04-b916-0b62e4d14ea0" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123401    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b740565b-db74-4f04-b916-0b62e4d14ea0" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123410    5051 cpu_manager.go:414] "containerMap: removing container" podUID="36b180c7-1e40-49fd-81a0-c73f93567bbc" containerName="payload-pre-processing"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123414    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36b180c7-1e40-49fd-81a0-c73f93567bbc" containerName="payload-pre-processing"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123421    5051 cpu_manager.go:414] "containerMap: removing container" podUID="8311401f-491c-43a6-a854-d3ae1f3255ae" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123425    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8311401f-491c-43a6-a854-d3ae1f3255ae" containerName="maas-api"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123432    5051 cpu_manager.go:414] "containerMap: removing container" podUID="f742538a-ef9d-48d6-a4c6-db2d7c311797" containerName="payload-processing"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123437    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f742538a-ef9d-48d6-a4c6-db2d7c311797" containerName="payload-processing"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123444    5051 cpu_manager.go:414] "containerMap: removing container" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123449    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123454    5051 cpu_manager.go:414] "containerMap: removing container" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123459    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123465    5051 cpu_manager.go:414] "containerMap: removing container" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123471    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123484    5051 cpu_manager.go:414] "containerMap: removing container" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerName="istio-proxy"
Nov 03 19:52:35.123470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123489    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123496    5051 cpu_manager.go:414] "containerMap: removing container" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123501    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123509    5051 cpu_manager.go:414] "containerMap: removing container" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123514    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123520    5051 cpu_manager.go:414] "containerMap: removing container" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123524    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123530    5051 cpu_manager.go:414] "containerMap: removing container" podUID="b6c6d13b-5904-4d07-8534-4b724313bb79" containerName="payload-processing"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123534    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6c6d13b-5904-4d07-8534-4b724313bb79" containerName="payload-processing"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123540    5051 cpu_manager.go:414] "containerMap: removing container" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123544    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123550    5051 cpu_manager.go:414] "containerMap: removing container" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123555    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123563    5051 cpu_manager.go:414] "containerMap: removing container" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123567    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123573    5051 cpu_manager.go:414] "containerMap: removing container" podUID="7ce60e5f-3d38-4e72-b857-bbdb675db1f5" containerName="payload-pre-processing"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123578    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ce60e5f-3d38-4e72-b857-bbdb675db1f5" containerName="payload-pre-processing"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123591    5051 cpu_manager.go:414] "containerMap: removing container" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123595    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123676    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="965226d2-5ae1-4bdf-97b3-6726e9c035d7" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123686    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123693    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="b740565b-db74-4f04-b916-0b62e4d14ea0" containerName="maas-api"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123700    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="36b180c7-1e40-49fd-81a0-c73f93567bbc" containerName="payload-pre-processing"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123707    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="8311401f-491c-43a6-a854-d3ae1f3255ae" containerName="maas-api"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123714    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="f742538a-ef9d-48d6-a4c6-db2d7c311797" containerName="payload-processing"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123721    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123727    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123732    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123738    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123744    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="a72d5d6f-7d72-4f4b-aba0-8c884a3723f8" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123750    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123755    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="8eade3a5-b467-4599-b909-cb3306407caa" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123762    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7655d6f-386a-4364-9343-6e56877d3faa" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123769    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6c6d13b-5904-4d07-8534-4b724313bb79" containerName="payload-processing"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123773    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="262d3eee-5594-4d75-83e2-1eb72166f710" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123778    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d11a037-b776-43f9-9d65-fa507737d381" containerName="main"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123784    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123788    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ce60e5f-3d38-4e72-b857-bbdb675db1f5" containerName="payload-pre-processing"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123796    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2ae4144-2795-4fee-b8c7-9a5859c55d35" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123802    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="526cdbab-900d-4990-9a52-deb0b1277d9a" containerName="maas-api"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123807    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="abcde2a0-cda0-4635-89f3-b791dee9537e" containerName="maas-api"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123812    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e1ad4ab-300a-425c-b752-bd6dd6108c4a" containerName="payload-pre-processing"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123818    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff0336ae-47dd-45db-9a74-1aaf1bbcb47d" containerName="payload-processing"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123824    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123829    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123836    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="4761a958-3fed-4c9f-aa46-314065cebb4a" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123840    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="082b8718-d755-4a33-9cc5-f1a157b846ec" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123846    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="3843274d-f1ec-4da5-806f-e841042b92f6" containerName="maas-api"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123851    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="dab38bfe-c49e-4031-8391-6075964ace09" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123857    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cf13e8c-1976-4218-943b-4deaf2dde7eb" containerName="maas-api"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123863    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="98ab7b99-b947-4052-81f6-dfe33dd18191" containerName="maas-api"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123868    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="610d96fe-f68f-4876-b85a-de3017180cf5" containerName="maas-api"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123873    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123878    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b0a399e-8e57-4d85-bdb5-94577f83a19c" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123883    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="deae5dcc-626c-4635-a8ff-455a48be49e1" containerName="istio-proxy"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123889    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a1a4117-0b43-4d57-8023-25bed714d499" containerName="payload-pre-processing"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123895    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c49123a-fa09-4756-8083-6d3a5c4d3f48" containerName="maas-api"
Nov 03 19:52:35.124859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.123899    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="82b74e21-57a7-489f-abe4-9814cf683aab" containerName="payload-pre-processing"
Nov 03 19:52:35.126943 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.126927    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:35.134747 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.134720    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 19:52:35.139291 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.135648    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dx9s4\""
Nov 03 19:52:35.139291 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.137175    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"]
Nov 03 19:52:35.170919 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.170889    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d72923e3-5mbv6\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:35.171074 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.170947    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d72923e3-5mbv6\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:35.171141 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.171030    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnfvb\" (UniqueName: \"kubernetes.io/projected/395bf147-d73d-4c0d-9693-93b6b8e15731-kube-api-access-wnfvb\") pod \"maas-api-revoke-sub-d72923e3-5mbv6\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:35.271891 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.271856    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d72923e3-5mbv6\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:35.272118 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.271927    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d72923e3-5mbv6\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:35.272118 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.272033    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnfvb\" (UniqueName: \"kubernetes.io/projected/395bf147-d73d-4c0d-9693-93b6b8e15731-kube-api-access-wnfvb\") pod \"maas-api-revoke-sub-d72923e3-5mbv6\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:35.272588 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.272561    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d72923e3-5mbv6\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:35.274530 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.274508    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d72923e3-5mbv6\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:35.284389 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.284363    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnfvb\" (UniqueName: \"kubernetes.io/projected/395bf147-d73d-4c0d-9693-93b6b8e15731-kube-api-access-wnfvb\") pod \"maas-api-revoke-sub-d72923e3-5mbv6\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:35.450100 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.449999    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:35.730282 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:35.730254    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"]
Nov 03 19:52:35.733349 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:52:35.733323    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395bf147_d73d_4c0d_9693_93b6b8e15731.slice/crio-9040d9da52bccd230148bed2cf66efabe9beb4c8384e9ee20b53c24f4cfcbecd WatchSource:0}: Error finding container 9040d9da52bccd230148bed2cf66efabe9beb4c8384e9ee20b53c24f4cfcbecd: Status 404 returned error can't find the container with id 9040d9da52bccd230148bed2cf66efabe9beb4c8384e9ee20b53c24f4cfcbecd
Nov 03 19:52:36.498103 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:36.498066    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"]
Nov 03 19:52:36.716661 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:36.716634    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6" event={"ID":"395bf147-d73d-4c0d-9693-93b6b8e15731","Type":"ContainerStarted","Data":"6923c0fa2fa7d4a1da682e4b18d6b8e0883321e44cbd8c483ab226a7fec7f0d5"}
Nov 03 19:52:36.716824 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:36.716684    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6" event={"ID":"395bf147-d73d-4c0d-9693-93b6b8e15731","Type":"ContainerStarted","Data":"9040d9da52bccd230148bed2cf66efabe9beb4c8384e9ee20b53c24f4cfcbecd"}
Nov 03 19:52:36.740028 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:36.739982    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6" podStartSLOduration=1.739966167 podStartE2EDuration="1.739966167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:52:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:52:36.738602164 +0000 UTC m=+1557.646624116" watchObservedRunningTime="2026-11-03 19:52:36.739966167 +0000 UTC m=+1557.647988103"
Nov 03 19:52:37.727319 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:37.727284    5051 generic.go:356] "Generic (PLEG): container finished" podID="395bf147-d73d-4c0d-9693-93b6b8e15731" containerID="6923c0fa2fa7d4a1da682e4b18d6b8e0883321e44cbd8c483ab226a7fec7f0d5" exitCode=0
Nov 03 19:52:37.727721 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:37.727393    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6" event={"ID":"395bf147-d73d-4c0d-9693-93b6b8e15731","Type":"ContainerDied","Data":"6923c0fa2fa7d4a1da682e4b18d6b8e0883321e44cbd8c483ab226a7fec7f0d5"}
Nov 03 19:52:38.464621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.464598    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:52:38.510836 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.510809    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f06217f-17b2-4745-bdf2-6ae876890f1a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f06217f-17b2-4745-bdf2-6ae876890f1a-istiod-ca-cert\") pod \"1f06217f-17b2-4745-bdf2-6ae876890f1a\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") "
Nov 03 19:52:38.511006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.510844    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-data\") pod \"1f06217f-17b2-4745-bdf2-6ae876890f1a\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") "
Nov 03 19:52:38.511006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.510893    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-envoy\") pod \"1f06217f-17b2-4745-bdf2-6ae876890f1a\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") "
Nov 03 19:52:38.511182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.511077    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-token\" (UniqueName: \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-token\") pod \"1f06217f-17b2-4745-bdf2-6ae876890f1a\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") "
Nov 03 19:52:38.511182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.511172    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-socket\") pod \"1f06217f-17b2-4745-bdf2-6ae876890f1a\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") "
Nov 03 19:52:38.511299 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.511204    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-data" pod "1f06217f-17b2-4745-bdf2-6ae876890f1a" (UID: "1f06217f-17b2-4745-bdf2-6ae876890f1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:38.511299 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.511221    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f06217f-17b2-4745-bdf2-6ae876890f1a-istiod-ca-cert" pod "1f06217f-17b2-4745-bdf2-6ae876890f1a" (UID: "1f06217f-17b2-4745-bdf2-6ae876890f1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:38.511299 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.511289    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-kube-api-access-k6dzx\" (UniqueName: \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-kube-api-access-k6dzx\") pod \"1f06217f-17b2-4745-bdf2-6ae876890f1a\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") "
Nov 03 19:52:38.511456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.511326    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-credential-socket\") pod \"1f06217f-17b2-4745-bdf2-6ae876890f1a\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") "
Nov 03 19:52:38.511456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.511410    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-socket" pod "1f06217f-17b2-4745-bdf2-6ae876890f1a" (UID: "1f06217f-17b2-4745-bdf2-6ae876890f1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:38.511456 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.511412    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-podinfo\") pod \"1f06217f-17b2-4745-bdf2-6ae876890f1a\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") "
Nov 03 19:52:38.511622 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.511464    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-certs\") pod \"1f06217f-17b2-4745-bdf2-6ae876890f1a\" (UID: \"1f06217f-17b2-4745-bdf2-6ae876890f1a\") "
Nov 03 19:52:38.511830 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.511794    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-credential-socket" pod "1f06217f-17b2-4745-bdf2-6ae876890f1a" (UID: "1f06217f-17b2-4745-bdf2-6ae876890f1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:38.511954 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.511929    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-certs" pod "1f06217f-17b2-4745-bdf2-6ae876890f1a" (UID: "1f06217f-17b2-4745-bdf2-6ae876890f1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:38.512151 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.512128    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:38.512241 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.512157    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:38.512241 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.512172    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f06217f-17b2-4745-bdf2-6ae876890f1a-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:38.512241 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.512186    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:38.512241 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.512199    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:38.513614 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.513584    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-envoy" pod "1f06217f-17b2-4745-bdf2-6ae876890f1a" (UID: "1f06217f-17b2-4745-bdf2-6ae876890f1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:38.513941 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.513914    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-podinfo" pod "1f06217f-17b2-4745-bdf2-6ae876890f1a" (UID: "1f06217f-17b2-4745-bdf2-6ae876890f1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:52:38.514035 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.514009    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-kube-api-access-k6dzx" pod "1f06217f-17b2-4745-bdf2-6ae876890f1a" (UID: "1f06217f-17b2-4745-bdf2-6ae876890f1a"). InnerVolumeSpecName "kube-api-access-k6dzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:38.514145 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.514124    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-token" pod "1f06217f-17b2-4745-bdf2-6ae876890f1a" (UID: "1f06217f-17b2-4745-bdf2-6ae876890f1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:38.613383 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.613358    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6dzx\" (UniqueName: \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-kube-api-access-k6dzx\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:38.613383 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.613381    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:38.613540 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.613392    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:38.613540 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.613402    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f06217f-17b2-4745-bdf2-6ae876890f1a-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:38.735634 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.735597    5051 generic.go:356] "Generic (PLEG): container finished" podID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerID="19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782" exitCode=0
Nov 03 19:52:38.736023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.735659    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"
Nov 03 19:52:38.736023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.735678    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" event={"ID":"1f06217f-17b2-4745-bdf2-6ae876890f1a","Type":"ContainerDied","Data":"19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782"}
Nov 03 19:52:38.736023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.735752    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg" event={"ID":"1f06217f-17b2-4745-bdf2-6ae876890f1a","Type":"ContainerDied","Data":"81f09ea5d1343fd6fe494ece60c13a0526af5bedce636135f801d52d6deaa711"}
Nov 03 19:52:38.736023 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.735774    5051 scope.go:122] "RemoveContainer" containerID="19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782"
Nov 03 19:52:38.746845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.746824    5051 scope.go:122] "RemoveContainer" containerID="4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2"
Nov 03 19:52:38.751897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.751877    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"]
Nov 03 19:52:38.757652 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.757635    5051 scope.go:122] "RemoveContainer" containerID="19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782"
Nov 03 19:52:38.757994 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:38.757973    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782\": container with ID starting with 19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782 not found: ID does not exist" containerID="19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782"
Nov 03 19:52:38.758085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.758001    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782"} err="failed to get container status \"19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782\": rpc error: code = NotFound desc = could not find container \"19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782\": container with ID starting with 19eae7cb89722a0302a0f0308b02d31e2d0f84ee08e7c34812a060e274b02782 not found: ID does not exist"
Nov 03 19:52:38.758085 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.758018    5051 scope.go:122] "RemoveContainer" containerID="4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2"
Nov 03 19:52:38.758327 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:38.758309    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2\": container with ID starting with 4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2 not found: ID does not exist" containerID="4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2"
Nov 03 19:52:38.758372 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.758334    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2"} err="failed to get container status \"4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2\": rpc error: code = NotFound desc = could not find container \"4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2\": container with ID starting with 4e19cfce7b634c8d273414b0347e19315e6c6e783072b97844c7b57a3da578c2 not found: ID does not exist"
Nov 03 19:52:38.771741 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.771714    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"]
Nov 03 19:52:38.782879 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.782857    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25944c-openshift-default-64b49f55b7-b8jgg"]
Nov 03 19:52:38.866690 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.866664    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:38.917827 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.917798    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-xks-ca\") pod \"395bf147-d73d-4c0d-9693-93b6b8e15731\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") "
Nov 03 19:52:38.917961 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.917903    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-openshift-ca\") pod \"395bf147-d73d-4c0d-9693-93b6b8e15731\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") "
Nov 03 19:52:38.918003 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.917978    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/395bf147-d73d-4c0d-9693-93b6b8e15731-kube-api-access-wnfvb\" (UniqueName: \"kubernetes.io/projected/395bf147-d73d-4c0d-9693-93b6b8e15731-kube-api-access-wnfvb\") pod \"395bf147-d73d-4c0d-9693-93b6b8e15731\" (UID: \"395bf147-d73d-4c0d-9693-93b6b8e15731\") "
Nov 03 19:52:38.918365 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.918331    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-openshift-ca" pod "395bf147-d73d-4c0d-9693-93b6b8e15731" (UID: "395bf147-d73d-4c0d-9693-93b6b8e15731"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:38.920484 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.920459    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-xks-ca" pod "395bf147-d73d-4c0d-9693-93b6b8e15731" (UID: "395bf147-d73d-4c0d-9693-93b6b8e15731"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:38.920552 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:38.920498    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/395bf147-d73d-4c0d-9693-93b6b8e15731-kube-api-access-wnfvb" pod "395bf147-d73d-4c0d-9693-93b6b8e15731" (UID: "395bf147-d73d-4c0d-9693-93b6b8e15731"). InnerVolumeSpecName "kube-api-access-wnfvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:39.019427 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:39.019361    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-openshift-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:39.019427 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:39.019388    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnfvb\" (UniqueName: \"kubernetes.io/projected/395bf147-d73d-4c0d-9693-93b6b8e15731-kube-api-access-wnfvb\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:39.019427 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:39.019398    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/395bf147-d73d-4c0d-9693-93b6b8e15731-maas-api-xks-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:39.228228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:39.228200    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" path="/var/lib/kubelet/pods/1f06217f-17b2-4745-bdf2-6ae876890f1a/volumes"
Nov 03 19:52:39.745101 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:39.745063    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6" event={"ID":"395bf147-d73d-4c0d-9693-93b6b8e15731","Type":"ContainerDied","Data":"9040d9da52bccd230148bed2cf66efabe9beb4c8384e9ee20b53c24f4cfcbecd"}
Nov 03 19:52:39.745101 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:39.745091    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"
Nov 03 19:52:39.745505 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:39.745125    5051 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9040d9da52bccd230148bed2cf66efabe9beb4c8384e9ee20b53c24f4cfcbecd"
Nov 03 19:52:40.864299 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:40.864259    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"]
Nov 03 19:52:40.864763 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:40.864532    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy" containerID="cri-o://5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808" gracePeriod=30
Nov 03 19:52:41.432208 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432177    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"]
Nov 03 19:52:41.432676 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432662    5051 cpu_manager.go:414] "containerMap: removing container" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy"
Nov 03 19:52:41.432731 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432677    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy"
Nov 03 19:52:41.432731 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432699    5051 cpu_manager.go:414] "containerMap: removing container" podUID="395bf147-d73d-4c0d-9693-93b6b8e15731" containerName="revoke-keys"
Nov 03 19:52:41.432731 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432704    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="395bf147-d73d-4c0d-9693-93b6b8e15731" containerName="revoke-keys"
Nov 03 19:52:41.432731 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432714    5051 cpu_manager.go:414] "containerMap: removing container" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy"
Nov 03 19:52:41.432731 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432719    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy"
Nov 03 19:52:41.432890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432742    5051 cpu_manager.go:414] "containerMap: removing container" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy"
Nov 03 19:52:41.432890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432747    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="425fcd7d-1420-41a8-89d4-503a9d259b89" containerName="istio-proxy"
Nov 03 19:52:41.432890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432828    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy"
Nov 03 19:52:41.432890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432838    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="395bf147-d73d-4c0d-9693-93b6b8e15731" containerName="revoke-keys"
Nov 03 19:52:41.432890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.432847    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f06217f-17b2-4745-bdf2-6ae876890f1a" containerName="istio-proxy"
Nov 03 19:52:41.437532 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.437515    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:41.448070 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.447981    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"]
Nov 03 19:52:41.498621 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.498553    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7" podUID="c0e39677-39a0-4a57-9672-dbf5d49e3168" containerName="payload-pre-processing" containerID="cri-o://14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0" gracePeriod=25
Nov 03 19:52:41.504797 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.504770    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 19:52:41.504900 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.504883    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dx9s4\""
Nov 03 19:52:41.548248 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.548225    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlb6n\" (UniqueName: \"kubernetes.io/projected/e5416a45-98f0-494e-b68e-0422ed8420e4-kube-api-access-nlb6n\") pod \"maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:41.548356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.548269    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:41.548356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.548291    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:41.649555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.649521    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlb6n\" (UniqueName: \"kubernetes.io/projected/e5416a45-98f0-494e-b68e-0422ed8420e4-kube-api-access-nlb6n\") pod \"maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:41.649748 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.649570    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:41.649748 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.649590    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:41.650277 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.650247    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:41.652513 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.652482    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:41.664350 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.664318    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlb6n\" (UniqueName: \"kubernetes.io/projected/e5416a45-98f0-494e-b68e-0422ed8420e4-kube-api-access-nlb6n\") pod \"maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:41.756563 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.756533    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:41.951945 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.951910    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main" containerID="cri-o://14b7d9141c7bbac6d22f5fe862e731b04ade0dccc71da1881a4edcf3862b9fcd" gracePeriod=45
Nov 03 19:52:41.970169 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.970030    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" containerID="cri-o://7b5a14a8cc745ebaf81b087822691f35c56c0573de5983a18bde35b1990518d4" gracePeriod=45
Nov 03 19:52:41.976980 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.976949    5051 patch_prober.go:29] interesting pod/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 19:52:41.977154 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:41.977029    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 19:52:42.042943 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.042916    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"]
Nov 03 19:52:42.061415 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:52:42.061381    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5416a45_98f0_494e_b68e_0422ed8420e4.slice/crio-b92c435ee40c302ed404048313005838a704bee51dab991338bc88239bb52ce3 WatchSource:0}: Error finding container b92c435ee40c302ed404048313005838a704bee51dab991338bc88239bb52ce3: Status 404 returned error can't find the container with id b92c435ee40c302ed404048313005838a704bee51dab991338bc88239bb52ce3
Nov 03 19:52:42.133670 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.133628    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": dial tcp 10.132.0.67:8000: connect: connection refused"
Nov 03 19:52:42.228331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.228300    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:52:42.244054 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.244023    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:52:42.358425 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.358398    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-dshm\") pod \"1e112a60-5672-426a-8788-810b9cab54a1\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") "
Nov 03 19:52:42.358549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.358442    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-model-cache\") pod \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") "
Nov 03 19:52:42.358549 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.358510    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e112a60-5672-426a-8788-810b9cab54a1-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e112a60-5672-426a-8788-810b9cab54a1-tls-certs\") pod \"1e112a60-5672-426a-8788-810b9cab54a1\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") "
Nov 03 19:52:42.358685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.358586    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdc62bab-9a46-4aac-81ef-21735bcc8e43-kube-api-access-4wmbb\" (UniqueName: \"kubernetes.io/projected/bdc62bab-9a46-4aac-81ef-21735bcc8e43-kube-api-access-4wmbb\") pod \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") "
Nov 03 19:52:42.358685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.358627    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e112a60-5672-426a-8788-810b9cab54a1-kube-api-access-h5kkg\" (UniqueName: \"kubernetes.io/projected/1e112a60-5672-426a-8788-810b9cab54a1-kube-api-access-h5kkg\") pod \"1e112a60-5672-426a-8788-810b9cab54a1\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") "
Nov 03 19:52:42.358685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.358656    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-model-cache\") pod \"1e112a60-5672-426a-8788-810b9cab54a1\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") "
Nov 03 19:52:42.358845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.358704    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tls-certs\") pod \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") "
Nov 03 19:52:42.358845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.358748    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-home\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-home\") pod \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") "
Nov 03 19:52:42.358845 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.358785    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-tmp-dir\") pod \"1e112a60-5672-426a-8788-810b9cab54a1\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") "
Nov 03 19:52:42.358998 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.358922    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-home\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-home\") pod \"1e112a60-5672-426a-8788-810b9cab54a1\" (UID: \"1e112a60-5672-426a-8788-810b9cab54a1\") "
Nov 03 19:52:42.358998 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.358962    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tmp-dir\") pod \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") "
Nov 03 19:52:42.359118 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.359011    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-dshm\") pod \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\" (UID: \"bdc62bab-9a46-4aac-81ef-21735bcc8e43\") "
Nov 03 19:52:42.359799 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.359776    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-model-cache" pod "1e112a60-5672-426a-8788-810b9cab54a1" (UID: "1e112a60-5672-426a-8788-810b9cab54a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:42.359960 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.359928    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-home" pod "bdc62bab-9a46-4aac-81ef-21735bcc8e43" (UID: "bdc62bab-9a46-4aac-81ef-21735bcc8e43"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:42.359960 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.359949    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-home" pod "1e112a60-5672-426a-8788-810b9cab54a1" (UID: "1e112a60-5672-426a-8788-810b9cab54a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:42.360114 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.359894    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-tmp-dir" pod "1e112a60-5672-426a-8788-810b9cab54a1" (UID: "1e112a60-5672-426a-8788-810b9cab54a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:42.360300 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.360272    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tmp-dir" pod "bdc62bab-9a46-4aac-81ef-21735bcc8e43" (UID: "bdc62bab-9a46-4aac-81ef-21735bcc8e43"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:42.360416 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.360343    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-model-cache" pod "bdc62bab-9a46-4aac-81ef-21735bcc8e43" (UID: "bdc62bab-9a46-4aac-81ef-21735bcc8e43"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:42.361711 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.361685    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-dshm" pod "bdc62bab-9a46-4aac-81ef-21735bcc8e43" (UID: "bdc62bab-9a46-4aac-81ef-21735bcc8e43"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:42.362114 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.362009    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e112a60-5672-426a-8788-810b9cab54a1-tls-certs" pod "1e112a60-5672-426a-8788-810b9cab54a1" (UID: "1e112a60-5672-426a-8788-810b9cab54a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:42.362507 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.362479    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tls-certs" pod "bdc62bab-9a46-4aac-81ef-21735bcc8e43" (UID: "bdc62bab-9a46-4aac-81ef-21735bcc8e43"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:42.362588 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.362489    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc62bab-9a46-4aac-81ef-21735bcc8e43-kube-api-access-4wmbb" pod "bdc62bab-9a46-4aac-81ef-21735bcc8e43" (UID: "bdc62bab-9a46-4aac-81ef-21735bcc8e43"). InnerVolumeSpecName "kube-api-access-4wmbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:42.363645 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.363625    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-dshm" pod "1e112a60-5672-426a-8788-810b9cab54a1" (UID: "1e112a60-5672-426a-8788-810b9cab54a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:42.363738 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.363706    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e112a60-5672-426a-8788-810b9cab54a1-kube-api-access-h5kkg" pod "1e112a60-5672-426a-8788-810b9cab54a1" (UID: "1e112a60-5672-426a-8788-810b9cab54a1"). InnerVolumeSpecName "kube-api-access-h5kkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:42.460989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.460959    5051 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-home\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.460989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.460986    5051 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tmp-dir\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.460989 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.460995    5051 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-dshm\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.461209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.461002    5051 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-dshm\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.461209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.461010    5051 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-model-cache\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.461209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.461019    5051 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e112a60-5672-426a-8788-810b9cab54a1-tls-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.461209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.461028    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wmbb\" (UniqueName: \"kubernetes.io/projected/bdc62bab-9a46-4aac-81ef-21735bcc8e43-kube-api-access-4wmbb\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.461209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.461036    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5kkg\" (UniqueName: \"kubernetes.io/projected/1e112a60-5672-426a-8788-810b9cab54a1-kube-api-access-h5kkg\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.461209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.461078    5051 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-model-cache\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.461209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.461086    5051 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc62bab-9a46-4aac-81ef-21735bcc8e43-tls-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.461209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.461094    5051 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc62bab-9a46-4aac-81ef-21735bcc8e43-home\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.461209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.461102    5051 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e112a60-5672-426a-8788-810b9cab54a1-tmp-dir\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:42.528729 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.528705    5051 scope.go:122] "RemoveContainer" containerID="14b7d9141c7bbac6d22f5fe862e731b04ade0dccc71da1881a4edcf3862b9fcd"
Nov 03 19:52:42.539380 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.539351    5051 scope.go:122] "RemoveContainer" containerID="7b5a14a8cc745ebaf81b087822691f35c56c0573de5983a18bde35b1990518d4"
Nov 03 19:52:42.768009 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.767977    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5" event={"ID":"e5416a45-98f0-494e-b68e-0422ed8420e4","Type":"ContainerStarted","Data":"6139637d206ee4ea1f1df03c079b9163ad8c552f55dac3e1f397bb96eae0f31d"}
Nov 03 19:52:42.768150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.768032    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5" event={"ID":"e5416a45-98f0-494e-b68e-0422ed8420e4","Type":"ContainerStarted","Data":"b92c435ee40c302ed404048313005838a704bee51dab991338bc88239bb52ce3"}
Nov 03 19:52:42.768150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.768074    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" event={"ID":"1e112a60-5672-426a-8788-810b9cab54a1","Type":"ContainerDied","Data":"7b5a14a8cc745ebaf81b087822691f35c56c0573de5983a18bde35b1990518d4"}
Nov 03 19:52:42.768150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.768090    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8" event={"ID":"1e112a60-5672-426a-8788-810b9cab54a1","Type":"ContainerDied","Data":"be65dc9a39717ebe4f6160fcbf0102efb50a3d3d534f6eb341c490213810b3a2"}
Nov 03 19:52:42.768150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.768115    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" event={"ID":"bdc62bab-9a46-4aac-81ef-21735bcc8e43","Type":"ContainerDied","Data":"14b7d9141c7bbac6d22f5fe862e731b04ade0dccc71da1881a4edcf3862b9fcd"}
Nov 03 19:52:42.768150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.768127    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4" event={"ID":"bdc62bab-9a46-4aac-81ef-21735bcc8e43","Type":"ContainerDied","Data":"fa1ca5ccbc5bd5ed9d2de2c83dc330edaed380da57036c1cfacf1cf84b9977da"}
Nov 03 19:52:42.768304 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.768169    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"
Nov 03 19:52:42.768304 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.768182    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"
Nov 03 19:52:42.791338 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.791297    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5" podStartSLOduration=1.7912834869999998 podStartE2EDuration="1.791283487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:52:42.788853665 +0000 UTC m=+1563.696875623" watchObservedRunningTime="2026-11-03 19:52:42.791283487 +0000 UTC m=+1563.699305425"
Nov 03 19:52:42.825475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.825448    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"]
Nov 03 19:52:42.828753 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.828731    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-05e217/simulator-w5-05e217-kserve-688bc8c868-jz8s8"]
Nov 03 19:52:42.859683 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.859623    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"]
Nov 03 19:52:42.868850 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:42.868823    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-05e217/distinct-w5-05e217-kserve-78f864878d-cvzj4"]
Nov 03 19:52:43.228753 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:43.228719    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e112a60-5672-426a-8788-810b9cab54a1" path="/var/lib/kubelet/pods/1e112a60-5672-426a-8788-810b9cab54a1/volumes"
Nov 03 19:52:43.229205 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:43.229189    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" path="/var/lib/kubelet/pods/bdc62bab-9a46-4aac-81ef-21735bcc8e43/volumes"
Nov 03 19:52:43.782402 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:43.782369    5051 generic.go:356] "Generic (PLEG): container finished" podID="e5416a45-98f0-494e-b68e-0422ed8420e4" containerID="6139637d206ee4ea1f1df03c079b9163ad8c552f55dac3e1f397bb96eae0f31d" exitCode=0
Nov 03 19:52:43.782611 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:43.782410    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5" event={"ID":"e5416a45-98f0-494e-b68e-0422ed8420e4","Type":"ContainerDied","Data":"6139637d206ee4ea1f1df03c079b9163ad8c552f55dac3e1f397bb96eae0f31d"}
Nov 03 19:52:44.932856 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:44.932834    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:45.091012 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.090933    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-openshift-ca\") pod \"e5416a45-98f0-494e-b68e-0422ed8420e4\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") "
Nov 03 19:52:45.091012 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.091004    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5416a45-98f0-494e-b68e-0422ed8420e4-kube-api-access-nlb6n\" (UniqueName: \"kubernetes.io/projected/e5416a45-98f0-494e-b68e-0422ed8420e4-kube-api-access-nlb6n\") pod \"e5416a45-98f0-494e-b68e-0422ed8420e4\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") "
Nov 03 19:52:45.091231 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.091146    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-xks-ca\") pod \"e5416a45-98f0-494e-b68e-0422ed8420e4\" (UID: \"e5416a45-98f0-494e-b68e-0422ed8420e4\") "
Nov 03 19:52:45.091362 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.091336    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-openshift-ca" pod "e5416a45-98f0-494e-b68e-0422ed8420e4" (UID: "e5416a45-98f0-494e-b68e-0422ed8420e4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:45.091517 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.091500    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-openshift-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:45.093186 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.093161    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-xks-ca" pod "e5416a45-98f0-494e-b68e-0422ed8420e4" (UID: "e5416a45-98f0-494e-b68e-0422ed8420e4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:45.093327 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.093160    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5416a45-98f0-494e-b68e-0422ed8420e4-kube-api-access-nlb6n" pod "e5416a45-98f0-494e-b68e-0422ed8420e4" (UID: "e5416a45-98f0-494e-b68e-0422ed8420e4"). InnerVolumeSpecName "kube-api-access-nlb6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:45.192494 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.192460    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5416a45-98f0-494e-b68e-0422ed8420e4-maas-api-xks-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:45.192494 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.192491    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlb6n\" (UniqueName: \"kubernetes.io/projected/e5416a45-98f0-494e-b68e-0422ed8420e4-kube-api-access-nlb6n\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:45.626475 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.626443    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"]
Nov 03 19:52:45.626946 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.626933    5051 cpu_manager.go:414] "containerMap: removing container" podUID="e5416a45-98f0-494e-b68e-0422ed8420e4" containerName="revoke-keys"
Nov 03 19:52:45.627006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.626948    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5416a45-98f0-494e-b68e-0422ed8420e4" containerName="revoke-keys"
Nov 03 19:52:45.627006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.626967    5051 cpu_manager.go:414] "containerMap: removing container" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main"
Nov 03 19:52:45.627006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.626972    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main"
Nov 03 19:52:45.627006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.626985    5051 cpu_manager.go:414] "containerMap: removing container" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main"
Nov 03 19:52:45.627006 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.626990    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main"
Nov 03 19:52:45.627194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.627083    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5416a45-98f0-494e-b68e-0422ed8420e4" containerName="revoke-keys"
Nov 03 19:52:45.627194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.627094    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdc62bab-9a46-4aac-81ef-21735bcc8e43" containerName="main"
Nov 03 19:52:45.627194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.627102    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e112a60-5672-426a-8788-810b9cab54a1" containerName="main"
Nov 03 19:52:45.630147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.630131    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:45.641386 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.640772    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"]
Nov 03 19:52:45.796136 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.796106    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"
Nov 03 19:52:45.796322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.796143    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5" event={"ID":"e5416a45-98f0-494e-b68e-0422ed8420e4","Type":"ContainerDied","Data":"b92c435ee40c302ed404048313005838a704bee51dab991338bc88239bb52ce3"}
Nov 03 19:52:45.796322 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.796196    5051 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b92c435ee40c302ed404048313005838a704bee51dab991338bc88239bb52ce3"
Nov 03 19:52:45.798284 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.798257    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66361ab1-lkws6\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:45.798393 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.798307    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66361ab1-lkws6\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:45.798501 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.798481    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hhjn\" (UniqueName: \"kubernetes.io/projected/30822646-6bb1-48e1-8a7f-07cab19a7b0d-kube-api-access-5hhjn\") pod \"maas-api-revoke-sub-66361ab1-lkws6\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:45.899032 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.899008    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66361ab1-lkws6\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:45.899143 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.899070    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66361ab1-lkws6\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:45.899213 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.899160    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hhjn\" (UniqueName: \"kubernetes.io/projected/30822646-6bb1-48e1-8a7f-07cab19a7b0d-kube-api-access-5hhjn\") pod \"maas-api-revoke-sub-66361ab1-lkws6\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:45.899614 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.899594    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66361ab1-lkws6\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:45.901288 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.901266    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66361ab1-lkws6\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:45.913700 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.913678    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hhjn\" (UniqueName: \"kubernetes.io/projected/30822646-6bb1-48e1-8a7f-07cab19a7b0d-kube-api-access-5hhjn\") pod \"maas-api-revoke-sub-66361ab1-lkws6\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:45.946695 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:45.946666    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:46.172136 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.172114    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:52:46.231339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.231310    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"]
Nov 03 19:52:46.233950 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:52:46.233922    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30822646_6bb1_48e1_8a7f_07cab19a7b0d.slice/crio-b84324b90cc095d0d4eed4f00bfda2d4a18418037f4c79bda01fd8f11090a82b WatchSource:0}: Error finding container b84324b90cc095d0d4eed4f00bfda2d4a18418037f4c79bda01fd8f11090a82b: Status 404 returned error can't find the container with id b84324b90cc095d0d4eed4f00bfda2d4a18418037f4c79bda01fd8f11090a82b
Nov 03 19:52:46.303689 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.303661    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-data\") pod \"7a258474-8531-448f-8e4a-b7e4b39728fc\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") "
Nov 03 19:52:46.303816 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.303723    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-podinfo\") pod \"7a258474-8531-448f-8e4a-b7e4b39728fc\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") "
Nov 03 19:52:46.303816 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.303763    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a258474-8531-448f-8e4a-b7e4b39728fc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a258474-8531-448f-8e4a-b7e4b39728fc-istiod-ca-cert\") pod \"7a258474-8531-448f-8e4a-b7e4b39728fc\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") "
Nov 03 19:52:46.303816 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.303785    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-socket\") pod \"7a258474-8531-448f-8e4a-b7e4b39728fc\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") "
Nov 03 19:52:46.303990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.303841    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-certs\") pod \"7a258474-8531-448f-8e4a-b7e4b39728fc\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") "
Nov 03 19:52:46.303990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.303873    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-token\" (UniqueName: \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-token\") pod \"7a258474-8531-448f-8e4a-b7e4b39728fc\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") "
Nov 03 19:52:46.303990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.303932    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-data" pod "7a258474-8531-448f-8e4a-b7e4b39728fc" (UID: "7a258474-8531-448f-8e4a-b7e4b39728fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:46.303990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.303956    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-kube-api-access-zq6sn\" (UniqueName: \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-kube-api-access-zq6sn\") pod \"7a258474-8531-448f-8e4a-b7e4b39728fc\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") "
Nov 03 19:52:46.303990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.303985    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-envoy\") pod \"7a258474-8531-448f-8e4a-b7e4b39728fc\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") "
Nov 03 19:52:46.304256 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.304021    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-credential-socket\") pod \"7a258474-8531-448f-8e4a-b7e4b39728fc\" (UID: \"7a258474-8531-448f-8e4a-b7e4b39728fc\") "
Nov 03 19:52:46.304256 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.304129    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-certs" pod "7a258474-8531-448f-8e4a-b7e4b39728fc" (UID: "7a258474-8531-448f-8e4a-b7e4b39728fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:46.304367 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.304250    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-socket" pod "7a258474-8531-448f-8e4a-b7e4b39728fc" (UID: "7a258474-8531-448f-8e4a-b7e4b39728fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:46.304435 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.304409    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-credential-socket" pod "7a258474-8531-448f-8e4a-b7e4b39728fc" (UID: "7a258474-8531-448f-8e4a-b7e4b39728fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:46.304560 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.304533    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:46.304678 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.304562    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:46.304678 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.304577    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:46.304678 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.304592    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:46.304944 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.304921    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a258474-8531-448f-8e4a-b7e4b39728fc-istiod-ca-cert" pod "7a258474-8531-448f-8e4a-b7e4b39728fc" (UID: "7a258474-8531-448f-8e4a-b7e4b39728fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:46.306110 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.306084    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-podinfo" pod "7a258474-8531-448f-8e4a-b7e4b39728fc" (UID: "7a258474-8531-448f-8e4a-b7e4b39728fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:52:46.306194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.306123    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-token" pod "7a258474-8531-448f-8e4a-b7e4b39728fc" (UID: "7a258474-8531-448f-8e4a-b7e4b39728fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:46.306194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.306132    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-envoy" pod "7a258474-8531-448f-8e4a-b7e4b39728fc" (UID: "7a258474-8531-448f-8e4a-b7e4b39728fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:46.306194 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.306168    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-kube-api-access-zq6sn" pod "7a258474-8531-448f-8e4a-b7e4b39728fc" (UID: "7a258474-8531-448f-8e4a-b7e4b39728fc"). InnerVolumeSpecName "kube-api-access-zq6sn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:46.405990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.405937    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:46.405990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.405958    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a258474-8531-448f-8e4a-b7e4b39728fc-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:46.405990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.405967    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:46.405990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.405976    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq6sn\" (UniqueName: \"kubernetes.io/projected/7a258474-8531-448f-8e4a-b7e4b39728fc-kube-api-access-zq6sn\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:46.405990 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.405985    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a258474-8531-448f-8e4a-b7e4b39728fc-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:46.803079 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.803011    5051 generic.go:356] "Generic (PLEG): container finished" podID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerID="5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808" exitCode=0
Nov 03 19:52:46.803232 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.803079    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" event={"ID":"7a258474-8531-448f-8e4a-b7e4b39728fc","Type":"ContainerDied","Data":"5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808"}
Nov 03 19:52:46.803232 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.803143    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2" event={"ID":"7a258474-8531-448f-8e4a-b7e4b39728fc","Type":"ContainerDied","Data":"74d3569bd0298a318ccf0afef619893f9c554c763f432d6ccf934130b76cf41d"}
Nov 03 19:52:46.803232 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.803165    5051 scope.go:122] "RemoveContainer" containerID="5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808"
Nov 03 19:52:46.803232 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.803191    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"
Nov 03 19:52:46.805859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.805654    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6" event={"ID":"30822646-6bb1-48e1-8a7f-07cab19a7b0d","Type":"ContainerStarted","Data":"33c1279e4820b8192d62172b7c8e0abaaec788bcf5b96728bef6e69e3961db2b"}
Nov 03 19:52:46.805859 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.805698    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6" event={"ID":"30822646-6bb1-48e1-8a7f-07cab19a7b0d","Type":"ContainerStarted","Data":"b84324b90cc095d0d4eed4f00bfda2d4a18418037f4c79bda01fd8f11090a82b"}
Nov 03 19:52:46.816423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.816400    5051 scope.go:122] "RemoveContainer" containerID="9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0"
Nov 03 19:52:46.826187 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.826129    5051 scope.go:122] "RemoveContainer" containerID="5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808"
Nov 03 19:52:46.826532 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:46.826510    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808\": container with ID starting with 5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808 not found: ID does not exist" containerID="5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808"
Nov 03 19:52:46.826612 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.826540    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808"} err="failed to get container status \"5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808\": rpc error: code = NotFound desc = could not find container \"5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808\": container with ID starting with 5dbc5aa67091383961218222354b96fb0de5800dd3b9305e39e6e73f653b1808 not found: ID does not exist"
Nov 03 19:52:46.826612 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.826576    5051 scope.go:122] "RemoveContainer" containerID="9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0"
Nov 03 19:52:46.826863 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:46.826849    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0\": container with ID starting with 9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0 not found: ID does not exist" containerID="9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0"
Nov 03 19:52:46.826904 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.826867    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0"} err="failed to get container status \"9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0\": rpc error: code = NotFound desc = could not find container \"9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0\": container with ID starting with 9a6c1c1b56ac2d595e5a4084b5c227ddf35991360588b96cfd9d5073cbcdf6a0 not found: ID does not exist"
Nov 03 19:52:46.840772 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.840720    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6" podStartSLOduration=1.840704125 podStartE2EDuration="1.840704125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:52:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:52:46.837451845 +0000 UTC m=+1567.745473785" watchObservedRunningTime="2026-11-03 19:52:46.840704125 +0000 UTC m=+1567.748726065"
Nov 03 19:52:46.864551 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.864522    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"]
Nov 03 19:52:46.869674 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:46.869645    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-c6a08e-openshift-default-7945f786df-8vrp2"]
Nov 03 19:52:47.228551 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:47.228515    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" path="/var/lib/kubelet/pods/7a258474-8531-448f-8e4a-b7e4b39728fc/volumes"
Nov 03 19:52:47.812523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:47.812494    5051 generic.go:356] "Generic (PLEG): container finished" podID="30822646-6bb1-48e1-8a7f-07cab19a7b0d" containerID="33c1279e4820b8192d62172b7c8e0abaaec788bcf5b96728bef6e69e3961db2b" exitCode=0
Nov 03 19:52:47.812697 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:47.812584    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6" event={"ID":"30822646-6bb1-48e1-8a7f-07cab19a7b0d","Type":"ContainerDied","Data":"33c1279e4820b8192d62172b7c8e0abaaec788bcf5b96728bef6e69e3961db2b"}
Nov 03 19:52:48.723510 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:48.723478    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:52:48.971628 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:48.971602    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:49.136306 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.136226    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30822646-6bb1-48e1-8a7f-07cab19a7b0d-kube-api-access-5hhjn\" (UniqueName: \"kubernetes.io/projected/30822646-6bb1-48e1-8a7f-07cab19a7b0d-kube-api-access-5hhjn\") pod \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") "
Nov 03 19:52:49.136470 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.136390    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-openshift-ca\") pod \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") "
Nov 03 19:52:49.136518 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.136470    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-xks-ca\") pod \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\" (UID: \"30822646-6bb1-48e1-8a7f-07cab19a7b0d\") "
Nov 03 19:52:49.136808 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.136783    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-openshift-ca" pod "30822646-6bb1-48e1-8a7f-07cab19a7b0d" (UID: "30822646-6bb1-48e1-8a7f-07cab19a7b0d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:49.136897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.136882    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-openshift-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:49.138438 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.138413    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-xks-ca" pod "30822646-6bb1-48e1-8a7f-07cab19a7b0d" (UID: "30822646-6bb1-48e1-8a7f-07cab19a7b0d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:49.138580 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.138493    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30822646-6bb1-48e1-8a7f-07cab19a7b0d-kube-api-access-5hhjn" pod "30822646-6bb1-48e1-8a7f-07cab19a7b0d" (UID: "30822646-6bb1-48e1-8a7f-07cab19a7b0d"). InnerVolumeSpecName "kube-api-access-5hhjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:49.237785 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.237764    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30822646-6bb1-48e1-8a7f-07cab19a7b0d-maas-api-xks-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:49.237879 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.237787    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hhjn\" (UniqueName: \"kubernetes.io/projected/30822646-6bb1-48e1-8a7f-07cab19a7b0d-kube-api-access-5hhjn\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:49.826209 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.826182    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"
Nov 03 19:52:49.826597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.826238    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6" event={"ID":"30822646-6bb1-48e1-8a7f-07cab19a7b0d","Type":"ContainerDied","Data":"b84324b90cc095d0d4eed4f00bfda2d4a18418037f4c79bda01fd8f11090a82b"}
Nov 03 19:52:49.826597 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.826330    5051 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b84324b90cc095d0d4eed4f00bfda2d4a18418037f4c79bda01fd8f11090a82b"
Nov 03 19:52:49.828473 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:49.828439    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7" podUID="c0e39677-39a0-4a57-9672-dbf5d49e3168" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 19:52:52.832911 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:52.832832    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"]
Nov 03 19:52:52.837284 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:52.837256    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w4-763a69-qgtf5"]
Nov 03 19:52:53.228962 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:53.228926    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5416a45-98f0-494e-b68e-0422ed8420e4" path="/var/lib/kubelet/pods/e5416a45-98f0-494e-b68e-0422ed8420e4/volumes"
Nov 03 19:52:53.774161 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:53.774124    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main" containerID="cri-o://438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0" gracePeriod=45
Nov 03 19:52:53.829841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:53.829810    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"]
Nov 03 19:52:53.830131 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:53.830082    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx" podUID="35da5897-e86e-4951-8199-1f50f4d6b2e4" containerName="maas-api" containerID="cri-o://0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e" gracePeriod=30
Nov 03 19:52:54.127267 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.127232    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:52:54.143675 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.143651    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:52:54.203067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.203018    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-model-cache\") pod \"b41aef14-5753-4120-8136-68d05f9f67f4\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") "
Nov 03 19:52:54.203244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.203107    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-maas-api-tls\") pod \"35da5897-e86e-4951-8199-1f50f4d6b2e4\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") "
Nov 03 19:52:54.203244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.203206    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35da5897-e86e-4951-8199-1f50f4d6b2e4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35da5897-e86e-4951-8199-1f50f4d6b2e4-openshift-service-ca\") pod \"35da5897-e86e-4951-8199-1f50f4d6b2e4\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") "
Nov 03 19:52:54.203244 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.203236    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35da5897-e86e-4951-8199-1f50f4d6b2e4-kube-api-access-87vc2\" (UniqueName: \"kubernetes.io/projected/35da5897-e86e-4951-8199-1f50f4d6b2e4-kube-api-access-87vc2\") pod \"35da5897-e86e-4951-8199-1f50f4d6b2e4\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") "
Nov 03 19:52:54.203644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.203304    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b41aef14-5753-4120-8136-68d05f9f67f4-kube-api-access-c8njt\" (UniqueName: \"kubernetes.io/projected/b41aef14-5753-4120-8136-68d05f9f67f4-kube-api-access-c8njt\") pod \"b41aef14-5753-4120-8136-68d05f9f67f4\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") "
Nov 03 19:52:54.203644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.203311    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-model-cache" pod "b41aef14-5753-4120-8136-68d05f9f67f4" (UID: "b41aef14-5753-4120-8136-68d05f9f67f4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:54.203644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.203345    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-home\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-home\") pod \"b41aef14-5753-4120-8136-68d05f9f67f4\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") "
Nov 03 19:52:54.203644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.203375    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-dshm\") pod \"b41aef14-5753-4120-8136-68d05f9f67f4\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") "
Nov 03 19:52:54.203644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.203433    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-tmp-dir\") pod \"b41aef14-5753-4120-8136-68d05f9f67f4\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") "
Nov 03 19:52:54.203644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.203464    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-metrics-certs\") pod \"35da5897-e86e-4951-8199-1f50f4d6b2e4\" (UID: \"35da5897-e86e-4951-8199-1f50f4d6b2e4\") "
Nov 03 19:52:54.203644 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.203497    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b41aef14-5753-4120-8136-68d05f9f67f4-tls-certs\" (UniqueName: \"kubernetes.io/secret/b41aef14-5753-4120-8136-68d05f9f67f4-tls-certs\") pod \"b41aef14-5753-4120-8136-68d05f9f67f4\" (UID: \"b41aef14-5753-4120-8136-68d05f9f67f4\") "
Nov 03 19:52:54.204387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.204112    5051 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-model-cache\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:54.204387 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.204112    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-tmp-dir" pod "b41aef14-5753-4120-8136-68d05f9f67f4" (UID: "b41aef14-5753-4120-8136-68d05f9f67f4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:54.208873 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.206388    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35da5897-e86e-4951-8199-1f50f4d6b2e4-openshift-service-ca" pod "35da5897-e86e-4951-8199-1f50f4d6b2e4" (UID: "35da5897-e86e-4951-8199-1f50f4d6b2e4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:52:54.208873 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.206863    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-metrics-certs" pod "35da5897-e86e-4951-8199-1f50f4d6b2e4" (UID: "35da5897-e86e-4951-8199-1f50f4d6b2e4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:54.208873 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.207339    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-home" pod "b41aef14-5753-4120-8136-68d05f9f67f4" (UID: "b41aef14-5753-4120-8136-68d05f9f67f4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:54.223422 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.223387    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b41aef14-5753-4120-8136-68d05f9f67f4-tls-certs" pod "b41aef14-5753-4120-8136-68d05f9f67f4" (UID: "b41aef14-5753-4120-8136-68d05f9f67f4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:54.226754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.225558    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b41aef14-5753-4120-8136-68d05f9f67f4-kube-api-access-c8njt" pod "b41aef14-5753-4120-8136-68d05f9f67f4" (UID: "b41aef14-5753-4120-8136-68d05f9f67f4"). InnerVolumeSpecName "kube-api-access-c8njt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:54.226754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.225905    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-dshm" pod "b41aef14-5753-4120-8136-68d05f9f67f4" (UID: "b41aef14-5753-4120-8136-68d05f9f67f4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:52:54.226754 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.226201    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35da5897-e86e-4951-8199-1f50f4d6b2e4-kube-api-access-87vc2" pod "35da5897-e86e-4951-8199-1f50f4d6b2e4" (UID: "35da5897-e86e-4951-8199-1f50f4d6b2e4"). InnerVolumeSpecName "kube-api-access-87vc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:52:54.227868 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.227840    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-maas-api-tls" pod "35da5897-e86e-4951-8199-1f50f4d6b2e4" (UID: "35da5897-e86e-4951-8199-1f50f4d6b2e4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:52:54.305694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.305620    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:54.305694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.305671    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35da5897-e86e-4951-8199-1f50f4d6b2e4-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:54.305694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.305682    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87vc2\" (UniqueName: \"kubernetes.io/projected/35da5897-e86e-4951-8199-1f50f4d6b2e4-kube-api-access-87vc2\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:54.305694 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.305692    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8njt\" (UniqueName: \"kubernetes.io/projected/b41aef14-5753-4120-8136-68d05f9f67f4-kube-api-access-c8njt\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:54.305945 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.305701    5051 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-home\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:54.305945 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.305709    5051 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-dshm\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:54.305945 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.305716    5051 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41aef14-5753-4120-8136-68d05f9f67f4-tmp-dir\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:54.305945 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.305724    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35da5897-e86e-4951-8199-1f50f4d6b2e4-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:54.305945 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.305731    5051 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41aef14-5753-4120-8136-68d05f9f67f4-tls-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:52:54.860371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.860327    5051 generic.go:356] "Generic (PLEG): container finished" podID="b41aef14-5753-4120-8136-68d05f9f67f4" containerID="438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0" exitCode=0
Nov 03 19:52:54.860555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.860399    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"
Nov 03 19:52:54.860555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.860408    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" event={"ID":"b41aef14-5753-4120-8136-68d05f9f67f4","Type":"ContainerDied","Data":"438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0"}
Nov 03 19:52:54.860555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.860456    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9" event={"ID":"b41aef14-5753-4120-8136-68d05f9f67f4","Type":"ContainerDied","Data":"af56af62c1160d9f96b3dd4ded15e51a6c4318b194188a89fcf498d5eadb7b5f"}
Nov 03 19:52:54.860555 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.860474    5051 scope.go:122] "RemoveContainer" containerID="438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0"
Nov 03 19:52:54.861965 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.861944    5051 generic.go:356] "Generic (PLEG): container finished" podID="35da5897-e86e-4951-8199-1f50f4d6b2e4" containerID="0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e" exitCode=0
Nov 03 19:52:54.862028 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.862011    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"
Nov 03 19:52:54.862084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.862018    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx" event={"ID":"35da5897-e86e-4951-8199-1f50f4d6b2e4","Type":"ContainerDied","Data":"0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e"}
Nov 03 19:52:54.862127 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.862105    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx" event={"ID":"35da5897-e86e-4951-8199-1f50f4d6b2e4","Type":"ContainerDied","Data":"5a7eafb8deaf525c189efa3b91e8ccf4cf1cae4da01a9b8f6f9d2f38cc592610"}
Nov 03 19:52:54.875636 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.875617    5051 scope.go:122] "RemoveContainer" containerID="438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0"
Nov 03 19:52:54.876131 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:54.876097    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0\": container with ID starting with 438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0 not found: ID does not exist" containerID="438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0"
Nov 03 19:52:54.876225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.876137    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0"} err="failed to get container status \"438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0\": rpc error: code = NotFound desc = could not find container \"438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0\": container with ID starting with 438ba5b2dc05e8b6c0a09ac3e1e843f07a4a06e9ab734b34b137398f9cec1dc0 not found: ID does not exist"
Nov 03 19:52:54.876225 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.876161    5051 scope.go:122] "RemoveContainer" containerID="0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e"
Nov 03 19:52:54.884846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.884820    5051 scope.go:122] "RemoveContainer" containerID="0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e"
Nov 03 19:52:54.885110 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:52:54.885088    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e\": container with ID starting with 0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e not found: ID does not exist" containerID="0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e"
Nov 03 19:52:54.885178 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.885118    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e"} err="failed to get container status \"0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e\": rpc error: code = NotFound desc = could not find container \"0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e\": container with ID starting with 0450f146580c0f0e8132b2f79d2302014c8922c88169c67382751ca6e106df5e not found: ID does not exist"
Nov 03 19:52:54.921347 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.921323    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"]
Nov 03 19:52:54.928717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.928697    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-05e217-5c5d9cc5b4-xc2xx"]
Nov 03 19:52:54.944586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.944564    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"]
Nov 03 19:52:54.952805 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:54.952782    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-435bd8/distinct-2-w2-435bd8-kserve-5496c5c8b6-8qfg9"]
Nov 03 19:52:55.228449 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:55.228416    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35da5897-e86e-4951-8199-1f50f4d6b2e4" path="/var/lib/kubelet/pods/35da5897-e86e-4951-8199-1f50f4d6b2e4/volumes"
Nov 03 19:52:55.228846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:55.228833    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" path="/var/lib/kubelet/pods/b41aef14-5753-4120-8136-68d05f9f67f4/volumes"
Nov 03 19:52:59.828230 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:52:59.828189    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7" podUID="c0e39677-39a0-4a57-9672-dbf5d49e3168" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 19:53:01.765236 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.765181    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:53:01.916504 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.916468    5051 generic.go:356] "Generic (PLEG): container finished" podID="c0e39677-39a0-4a57-9672-dbf5d49e3168" containerID="14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0" exitCode=0
Nov 03 19:53:01.916654 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.916513    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"
Nov 03 19:53:01.916654 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.916543    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7" event={"ID":"c0e39677-39a0-4a57-9672-dbf5d49e3168","Type":"ContainerDied","Data":"14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0"}
Nov 03 19:53:01.916654 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.916600    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7" event={"ID":"c0e39677-39a0-4a57-9672-dbf5d49e3168","Type":"ContainerDied","Data":"eeb1bc0b06de721d53d64567fcc490d48dbbb519f32f168f77866ea281ccb8b3"}
Nov 03 19:53:01.916654 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.916629    5051 scope.go:122] "RemoveContainer" containerID="14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0"
Nov 03 19:53:01.920028 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.920005    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0e39677-39a0-4a57-9672-dbf5d49e3168-kube-api-access-8qwlx\" (UniqueName: \"kubernetes.io/projected/c0e39677-39a0-4a57-9672-dbf5d49e3168-kube-api-access-8qwlx\") pod \"c0e39677-39a0-4a57-9672-dbf5d49e3168\" (UID: \"c0e39677-39a0-4a57-9672-dbf5d49e3168\") "
Nov 03 19:53:01.920172 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.920064    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0e39677-39a0-4a57-9672-dbf5d49e3168-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e39677-39a0-4a57-9672-dbf5d49e3168-plugins-config-volume\") pod \"c0e39677-39a0-4a57-9672-dbf5d49e3168\" (UID: \"c0e39677-39a0-4a57-9672-dbf5d49e3168\") "
Nov 03 19:53:01.920751 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.920728    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0e39677-39a0-4a57-9672-dbf5d49e3168-plugins-config-volume" pod "c0e39677-39a0-4a57-9672-dbf5d49e3168" (UID: "c0e39677-39a0-4a57-9672-dbf5d49e3168"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:53:01.922870 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.922848    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0e39677-39a0-4a57-9672-dbf5d49e3168-kube-api-access-8qwlx" pod "c0e39677-39a0-4a57-9672-dbf5d49e3168" (UID: "c0e39677-39a0-4a57-9672-dbf5d49e3168"). InnerVolumeSpecName "kube-api-access-8qwlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:53:01.926791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.926772    5051 scope.go:122] "RemoveContainer" containerID="14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0"
Nov 03 19:53:01.927088 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:53:01.927034    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0\": container with ID starting with 14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0 not found: ID does not exist" containerID="14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0"
Nov 03 19:53:01.927207 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:01.927091    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0"} err="failed to get container status \"14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0\": rpc error: code = NotFound desc = could not find container \"14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0\": container with ID starting with 14443107dc22db131a570099e9551826a0aa0e80f5eb261df2a83e2116d5bfb0 not found: ID does not exist"
Nov 03 19:53:02.021547 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:02.021492    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qwlx\" (UniqueName: \"kubernetes.io/projected/c0e39677-39a0-4a57-9672-dbf5d49e3168-kube-api-access-8qwlx\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:02.021547 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:02.021515    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e39677-39a0-4a57-9672-dbf5d49e3168-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:02.239748 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:02.239715    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"]
Nov 03 19:53:02.241824 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:02.241800    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-036184-545bd5dc55-czwx7"]
Nov 03 19:53:02.767828 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:53:02.767792    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:53:03.235755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:03.235711    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0e39677-39a0-4a57-9672-dbf5d49e3168" path="/var/lib/kubelet/pods/c0e39677-39a0-4a57-9672-dbf5d49e3168/volumes"
Nov 03 19:53:03.973849 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:03.973812    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"]
Nov 03 19:53:03.974290 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:03.974107    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy" containerID="cri-o://ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8" gracePeriod=30
Nov 03 19:53:07.517256 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:07.517206    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"]
Nov 03 19:53:07.518024 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:07.517992    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy" containerID="cri-o://2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03" gracePeriod=30
Nov 03 19:53:09.262669 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.262644    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:53:09.409341 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409307    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-token\" (UniqueName: \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-token\") pod \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") "
Nov 03 19:53:09.409341 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409345    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istiod-ca-cert\") pod \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") "
Nov 03 19:53:09.409573 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409372    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-kube-api-access-d6nbh\" (UniqueName: \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-kube-api-access-d6nbh\") pod \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") "
Nov 03 19:53:09.409573 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409418    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-podinfo\") pod \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") "
Nov 03 19:53:09.409573 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409475    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-credential-socket\") pod \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") "
Nov 03 19:53:09.409573 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409503    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-certs\") pod \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") "
Nov 03 19:53:09.409783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409592    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-envoy\") pod \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") "
Nov 03 19:53:09.409783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409626    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-data\") pod \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") "
Nov 03 19:53:09.409783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409678    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-socket\") pod \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\" (UID: \"332ebb89-3bbd-47f3-a47e-33138a5bc16e\") "
Nov 03 19:53:09.409783 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409714    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istiod-ca-cert" pod "332ebb89-3bbd-47f3-a47e-33138a5bc16e" (UID: "332ebb89-3bbd-47f3-a47e-33138a5bc16e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:53:09.410011 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409921    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-data" pod "332ebb89-3bbd-47f3-a47e-33138a5bc16e" (UID: "332ebb89-3bbd-47f3-a47e-33138a5bc16e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:09.410011 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.409944    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-credential-socket" pod "332ebb89-3bbd-47f3-a47e-33138a5bc16e" (UID: "332ebb89-3bbd-47f3-a47e-33138a5bc16e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:09.410233 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.410152    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-certs" pod "332ebb89-3bbd-47f3-a47e-33138a5bc16e" (UID: "332ebb89-3bbd-47f3-a47e-33138a5bc16e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:09.410233 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.410163    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-socket" pod "332ebb89-3bbd-47f3-a47e-33138a5bc16e" (UID: "332ebb89-3bbd-47f3-a47e-33138a5bc16e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:09.410407 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.410285    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:09.410407 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.410297    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:09.410407 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.410307    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:09.410407 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.410320    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:09.410407 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.410330    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:09.411945 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.411922    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-podinfo" pod "332ebb89-3bbd-47f3-a47e-33138a5bc16e" (UID: "332ebb89-3bbd-47f3-a47e-33138a5bc16e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:53:09.412197 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.412171    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-envoy" pod "332ebb89-3bbd-47f3-a47e-33138a5bc16e" (UID: "332ebb89-3bbd-47f3-a47e-33138a5bc16e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:09.412197 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.412192    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-kube-api-access-d6nbh" pod "332ebb89-3bbd-47f3-a47e-33138a5bc16e" (UID: "332ebb89-3bbd-47f3-a47e-33138a5bc16e"). InnerVolumeSpecName "kube-api-access-d6nbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:53:09.412371 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.412209    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-token" pod "332ebb89-3bbd-47f3-a47e-33138a5bc16e" (UID: "332ebb89-3bbd-47f3-a47e-33138a5bc16e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:53:09.511363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.511294    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:09.511363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.511320    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:09.511363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.511329    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:09.511363 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.511338    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6nbh\" (UniqueName: \"kubernetes.io/projected/332ebb89-3bbd-47f3-a47e-33138a5bc16e-kube-api-access-d6nbh\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:09.969743 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.969709    5051 generic.go:356] "Generic (PLEG): container finished" podID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerID="ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8" exitCode=0
Nov 03 19:53:09.969904 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.969751    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" event={"ID":"332ebb89-3bbd-47f3-a47e-33138a5bc16e","Type":"ContainerDied","Data":"ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8"}
Nov 03 19:53:09.969904 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.969796    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" event={"ID":"332ebb89-3bbd-47f3-a47e-33138a5bc16e","Type":"ContainerDied","Data":"2cb5466fef359eaf1e9caa2b7d2b2b87c3d0841af2c2163b1f8b508d4e7d311b"}
Nov 03 19:53:09.969904 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.969810    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"
Nov 03 19:53:09.970013 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.969818    5051 scope.go:122] "RemoveContainer" containerID="ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8"
Nov 03 19:53:09.979935 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.979919    5051 scope.go:122] "RemoveContainer" containerID="a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364"
Nov 03 19:53:09.989007 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.988988    5051 scope.go:122] "RemoveContainer" containerID="ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8"
Nov 03 19:53:09.989296 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:53:09.989275    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8\": container with ID starting with ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8 not found: ID does not exist" containerID="ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8"
Nov 03 19:53:09.989351 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.989305    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8"} err="failed to get container status \"ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8\": rpc error: code = NotFound desc = could not find container \"ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8\": container with ID starting with ddde695b8fadbfb6fc44bc2a577a69234cff9c5a688c5e89a5cf5ffad0a1f7c8 not found: ID does not exist"
Nov 03 19:53:09.989351 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.989322    5051 scope.go:122] "RemoveContainer" containerID="a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364"
Nov 03 19:53:09.989562 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:53:09.989543    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364\": container with ID starting with a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364 not found: ID does not exist" containerID="a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364"
Nov 03 19:53:09.989601 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.989569    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364"} err="failed to get container status \"a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364\": rpc error: code = NotFound desc = could not find container \"a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364\": container with ID starting with a6388bd7256225cceef293bc0955fd4807ed341776df73ebeac4303faef3e364 not found: ID does not exist"
Nov 03 19:53:09.994986 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.994941    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"]
Nov 03 19:53:09.998245 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:09.998225    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59"]
Nov 03 19:53:10.167016 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:10.166971    5051 patch_prober.go:29] interesting pod/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:53:10.167182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:10.167051    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-036184-openshift-default-7bb76776f9-t9q59" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:53:11.228742 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:11.228707    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" path="/var/lib/kubelet/pods/332ebb89-3bbd-47f3-a47e-33138a5bc16e/volumes"
Nov 03 19:53:12.794650 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.794591    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:53:12.949443 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.949094    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-token\" (UniqueName: \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-token\") pod \"7d291fb3-9492-4b4a-9495-e55d85d7480d\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") "
Nov 03 19:53:12.949703 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.949673    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d291fb3-9492-4b4a-9495-e55d85d7480d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d291fb3-9492-4b4a-9495-e55d85d7480d-istiod-ca-cert\") pod \"7d291fb3-9492-4b4a-9495-e55d85d7480d\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") "
Nov 03 19:53:12.949905 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.949780    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-kube-api-access-rb5zt\" (UniqueName: \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-kube-api-access-rb5zt\") pod \"7d291fb3-9492-4b4a-9495-e55d85d7480d\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") "
Nov 03 19:53:12.949905 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.949841    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-socket\") pod \"7d291fb3-9492-4b4a-9495-e55d85d7480d\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") "
Nov 03 19:53:12.949905 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.949877    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-envoy\") pod \"7d291fb3-9492-4b4a-9495-e55d85d7480d\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") "
Nov 03 19:53:12.950250 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.949946    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-certs\") pod \"7d291fb3-9492-4b4a-9495-e55d85d7480d\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") "
Nov 03 19:53:12.950250 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.950021    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-podinfo\") pod \"7d291fb3-9492-4b4a-9495-e55d85d7480d\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") "
Nov 03 19:53:12.950250 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.950066    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-data\") pod \"7d291fb3-9492-4b4a-9495-e55d85d7480d\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") "
Nov 03 19:53:12.950250 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.950098    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-credential-socket\") pod \"7d291fb3-9492-4b4a-9495-e55d85d7480d\" (UID: \"7d291fb3-9492-4b4a-9495-e55d85d7480d\") "
Nov 03 19:53:12.950865 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.950699    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-credential-socket" pod "7d291fb3-9492-4b4a-9495-e55d85d7480d" (UID: "7d291fb3-9492-4b4a-9495-e55d85d7480d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:12.952246 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.952215    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-socket" pod "7d291fb3-9492-4b4a-9495-e55d85d7480d" (UID: "7d291fb3-9492-4b4a-9495-e55d85d7480d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:12.952570 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.952543    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-certs" pod "7d291fb3-9492-4b4a-9495-e55d85d7480d" (UID: "7d291fb3-9492-4b4a-9495-e55d85d7480d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:12.952739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.952712    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d291fb3-9492-4b4a-9495-e55d85d7480d-istiod-ca-cert" pod "7d291fb3-9492-4b4a-9495-e55d85d7480d" (UID: "7d291fb3-9492-4b4a-9495-e55d85d7480d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:53:12.952866 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.952782    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-data" pod "7d291fb3-9492-4b4a-9495-e55d85d7480d" (UID: "7d291fb3-9492-4b4a-9495-e55d85d7480d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:12.953544 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.953514    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-envoy" pod "7d291fb3-9492-4b4a-9495-e55d85d7480d" (UID: "7d291fb3-9492-4b4a-9495-e55d85d7480d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:12.953720 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.953698    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-podinfo" pod "7d291fb3-9492-4b4a-9495-e55d85d7480d" (UID: "7d291fb3-9492-4b4a-9495-e55d85d7480d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:53:12.953897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.953870    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-token" pod "7d291fb3-9492-4b4a-9495-e55d85d7480d" (UID: "7d291fb3-9492-4b4a-9495-e55d85d7480d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:53:12.953997 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.953908    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-kube-api-access-rb5zt" pod "7d291fb3-9492-4b4a-9495-e55d85d7480d" (UID: "7d291fb3-9492-4b4a-9495-e55d85d7480d"). InnerVolumeSpecName "kube-api-access-rb5zt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:53:12.990796 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.990763    5051 generic.go:356] "Generic (PLEG): container finished" podID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerID="2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03" exitCode=0
Nov 03 19:53:12.990925 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.990814    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"
Nov 03 19:53:12.990925 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.990843    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" event={"ID":"7d291fb3-9492-4b4a-9495-e55d85d7480d","Type":"ContainerDied","Data":"2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03"}
Nov 03 19:53:12.990925 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.990897    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8" event={"ID":"7d291fb3-9492-4b4a-9495-e55d85d7480d","Type":"ContainerDied","Data":"24ae5d04fcdf2faea4a489dbf4e5ab136a165e385e0f73975755006b6e1f9257"}
Nov 03 19:53:12.990925 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:12.990919    5051 scope.go:122] "RemoveContainer" containerID="2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03"
Nov 03 19:53:13.002710 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.002689    5051 scope.go:122] "RemoveContainer" containerID="2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03"
Nov 03 19:53:13.003014 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:53:13.002993    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03\": container with ID starting with 2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03 not found: ID does not exist" containerID="2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03"
Nov 03 19:53:13.003103 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.003026    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03"} err="failed to get container status \"2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03\": rpc error: code = NotFound desc = could not find container \"2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03\": container with ID starting with 2118bbe6fd630fdc616c7991b1160b533c56b01d18f22bd7e91de81d31861d03 not found: ID does not exist"
Nov 03 19:53:13.020345 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.020318    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"]
Nov 03 19:53:13.030844 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.030818    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-435bd8-openshift-default-6698b8f7c-fs8k8"]
Nov 03 19:53:13.051399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.051336    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d291fb3-9492-4b4a-9495-e55d85d7480d-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:13.051399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.051361    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rb5zt\" (UniqueName: \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-kube-api-access-rb5zt\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:13.051399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.051377    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:13.051399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.051388    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:13.051399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.051397    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:13.051399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.051405    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:13.051846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.051413    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:13.051846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.051421    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d291fb3-9492-4b4a-9495-e55d85d7480d-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:13.051846 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.051430    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d291fb3-9492-4b4a-9495-e55d85d7480d-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:13.228841 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:13.228800    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" path="/var/lib/kubelet/pods/7d291fb3-9492-4b4a-9495-e55d85d7480d/volumes"
Nov 03 19:53:19.132331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:19.132297    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:53:21.148852 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:21.148818    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"]
Nov 03 19:53:21.149354 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:21.149018    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy" containerID="cri-o://6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481" gracePeriod=30
Nov 03 19:53:26.403877 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.403845    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:53:26.483466 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.483435    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-certs\") pod \"701528a6-3102-4145-8c38-c31e6f80cd9f\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") "
Nov 03 19:53:26.483655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.483523    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/701528a6-3102-4145-8c38-c31e6f80cd9f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/701528a6-3102-4145-8c38-c31e6f80cd9f-istiod-ca-cert\") pod \"701528a6-3102-4145-8c38-c31e6f80cd9f\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") "
Nov 03 19:53:26.483655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.483562    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-kube-api-access-p5nfs\" (UniqueName: \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-kube-api-access-p5nfs\") pod \"701528a6-3102-4145-8c38-c31e6f80cd9f\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") "
Nov 03 19:53:26.483655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.483591    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-credential-socket\") pod \"701528a6-3102-4145-8c38-c31e6f80cd9f\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") "
Nov 03 19:53:26.483655 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.483626    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-socket\") pod \"701528a6-3102-4145-8c38-c31e6f80cd9f\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") "
Nov 03 19:53:26.483898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.483663    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-data\") pod \"701528a6-3102-4145-8c38-c31e6f80cd9f\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") "
Nov 03 19:53:26.483898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.483720    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-token\" (UniqueName: \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-token\") pod \"701528a6-3102-4145-8c38-c31e6f80cd9f\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") "
Nov 03 19:53:26.483898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.483753    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-envoy\") pod \"701528a6-3102-4145-8c38-c31e6f80cd9f\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") "
Nov 03 19:53:26.483898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.483794    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-podinfo\") pod \"701528a6-3102-4145-8c38-c31e6f80cd9f\" (UID: \"701528a6-3102-4145-8c38-c31e6f80cd9f\") "
Nov 03 19:53:26.484147 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.484085    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-credential-socket" pod "701528a6-3102-4145-8c38-c31e6f80cd9f" (UID: "701528a6-3102-4145-8c38-c31e6f80cd9f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:26.484284 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.484263    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:26.485331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.484691    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/701528a6-3102-4145-8c38-c31e6f80cd9f-istiod-ca-cert" pod "701528a6-3102-4145-8c38-c31e6f80cd9f" (UID: "701528a6-3102-4145-8c38-c31e6f80cd9f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:53:26.485331 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.485189    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-certs" pod "701528a6-3102-4145-8c38-c31e6f80cd9f" (UID: "701528a6-3102-4145-8c38-c31e6f80cd9f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:26.485466 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.485328    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-data" pod "701528a6-3102-4145-8c38-c31e6f80cd9f" (UID: "701528a6-3102-4145-8c38-c31e6f80cd9f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:26.485466 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.485387    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-socket" pod "701528a6-3102-4145-8c38-c31e6f80cd9f" (UID: "701528a6-3102-4145-8c38-c31e6f80cd9f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:26.486685 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.486601    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-podinfo" pod "701528a6-3102-4145-8c38-c31e6f80cd9f" (UID: "701528a6-3102-4145-8c38-c31e6f80cd9f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 19:53:26.487031 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.487000    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-kube-api-access-p5nfs" pod "701528a6-3102-4145-8c38-c31e6f80cd9f" (UID: "701528a6-3102-4145-8c38-c31e6f80cd9f"). InnerVolumeSpecName "kube-api-access-p5nfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:53:26.487738 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.487714    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-envoy" pod "701528a6-3102-4145-8c38-c31e6f80cd9f" (UID: "701528a6-3102-4145-8c38-c31e6f80cd9f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:26.489134 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.489109    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-token" pod "701528a6-3102-4145-8c38-c31e6f80cd9f" (UID: "701528a6-3102-4145-8c38-c31e6f80cd9f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:53:26.505265 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.505241    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"]
Nov 03 19:53:26.520369 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.520344    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"]
Nov 03 19:53:26.585847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.585817    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/701528a6-3102-4145-8c38-c31e6f80cd9f-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:26.585847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.585843    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5nfs\" (UniqueName: \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-kube-api-access-p5nfs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:26.585995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.585854    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:26.585995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.585863    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:26.585995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.585872    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:26.585995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.585880    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:26.585995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.585889    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/701528a6-3102-4145-8c38-c31e6f80cd9f-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:26.585995 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:26.585897    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/701528a6-3102-4145-8c38-c31e6f80cd9f-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:27.079550 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.079518    5051 generic.go:356] "Generic (PLEG): container finished" podID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerID="6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481" exitCode=0
Nov 03 19:53:27.079772 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.079585    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"
Nov 03 19:53:27.079772 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.079597    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" event={"ID":"701528a6-3102-4145-8c38-c31e6f80cd9f","Type":"ContainerDied","Data":"6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481"}
Nov 03 19:53:27.079772 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.079688    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb" event={"ID":"701528a6-3102-4145-8c38-c31e6f80cd9f","Type":"ContainerDied","Data":"fdd94bc608fe1c4e9d9cf59b8d033230b62601de836e1588b8e1a1b2af720928"}
Nov 03 19:53:27.079772 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.079718    5051 scope.go:122] "RemoveContainer" containerID="6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481"
Nov 03 19:53:27.090683 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.090663    5051 scope.go:122] "RemoveContainer" containerID="acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318"
Nov 03 19:53:27.098937 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.098917    5051 scope.go:122] "RemoveContainer" containerID="6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481"
Nov 03 19:53:27.099223 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:53:27.099200    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481\": container with ID starting with 6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481 not found: ID does not exist" containerID="6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481"
Nov 03 19:53:27.099273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.099230    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481"} err="failed to get container status \"6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481\": rpc error: code = NotFound desc = could not find container \"6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481\": container with ID starting with 6ac5975bd752d15c9197f6935d9d4645331dee7c46ee8be1a38a8373353ed481 not found: ID does not exist"
Nov 03 19:53:27.099273 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.099247    5051 scope.go:122] "RemoveContainer" containerID="acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318"
Nov 03 19:53:27.099494 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:53:27.099480    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318\": container with ID starting with acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318 not found: ID does not exist" containerID="acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318"
Nov 03 19:53:27.099532 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.099499    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318"} err="failed to get container status \"acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318\": rpc error: code = NotFound desc = could not find container \"acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318\": container with ID starting with acc5173d99b4580de0bba94a06a2131a48aadf66271cb38d7255fd59b7d9a318 not found: ID does not exist"
Nov 03 19:53:27.104221 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.104199    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"]
Nov 03 19:53:27.109923 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.109900    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-64454a-openshift-default-84c845c549-ndcdb"]
Nov 03 19:53:27.229641 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:27.229618    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" path="/var/lib/kubelet/pods/701528a6-3102-4145-8c38-c31e6f80cd9f/volumes"
Nov 03 19:53:31.823830 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:31.823796    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"]
Nov 03 19:53:32.766815 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:53:32.766776    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:53:32.971930 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.971896    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"]
Nov 03 19:53:32.972382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972367    5051 cpu_manager.go:414] "containerMap: removing container" podUID="30822646-6bb1-48e1-8a7f-07cab19a7b0d" containerName="revoke-keys"
Nov 03 19:53:32.972431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972383    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30822646-6bb1-48e1-8a7f-07cab19a7b0d" containerName="revoke-keys"
Nov 03 19:53:32.972431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972394    5051 cpu_manager.go:414] "containerMap: removing container" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy"
Nov 03 19:53:32.972431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972399    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy"
Nov 03 19:53:32.972431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972410    5051 cpu_manager.go:414] "containerMap: removing container" podUID="35da5897-e86e-4951-8199-1f50f4d6b2e4" containerName="maas-api"
Nov 03 19:53:32.972431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972415    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35da5897-e86e-4951-8199-1f50f4d6b2e4" containerName="maas-api"
Nov 03 19:53:32.972431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972426    5051 cpu_manager.go:414] "containerMap: removing container" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main"
Nov 03 19:53:32.972431 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972431    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972439    5051 cpu_manager.go:414] "containerMap: removing container" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972444    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972452    5051 cpu_manager.go:414] "containerMap: removing container" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972457    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972474    5051 cpu_manager.go:414] "containerMap: removing container" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972479    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972490    5051 cpu_manager.go:414] "containerMap: removing container" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972496    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972505    5051 cpu_manager.go:414] "containerMap: removing container" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972509    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972517    5051 cpu_manager.go:414] "containerMap: removing container" podUID="c0e39677-39a0-4a57-9672-dbf5d49e3168" containerName="payload-pre-processing"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972521    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0e39677-39a0-4a57-9672-dbf5d49e3168" containerName="payload-pre-processing"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972583    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972593    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0e39677-39a0-4a57-9672-dbf5d49e3168" containerName="payload-pre-processing"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972600    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="30822646-6bb1-48e1-8a7f-07cab19a7b0d" containerName="revoke-keys"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972606    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972611    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="35da5897-e86e-4951-8199-1f50f4d6b2e4" containerName="maas-api"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972617    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="b41aef14-5753-4120-8136-68d05f9f67f4" containerName="main"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972622    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972628    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d291fb3-9492-4b4a-9495-e55d85d7480d" containerName="istio-proxy"
Nov 03 19:53:32.972637 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.972636    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy"
Nov 03 19:53:32.975789 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.975773    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:32.981057 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.981019    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dx9s4\""
Nov 03 19:53:32.982250 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.982230    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 19:53:32.985982 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:32.985958    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"]
Nov 03 19:53:33.046890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:33.046829    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs7p7\" (UniqueName: \"kubernetes.io/projected/65043676-48c0-4ac1-8dba-934aea956a08-kube-api-access-gs7p7\") pod \"maas-api-revoke-sub-2eb54fff-tcql2\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:33.046890 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:33.046869    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2eb54fff-tcql2\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:33.047092 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:33.046901    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2eb54fff-tcql2\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:33.148254 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:33.148220    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7p7\" (UniqueName: \"kubernetes.io/projected/65043676-48c0-4ac1-8dba-934aea956a08-kube-api-access-gs7p7\") pod \"maas-api-revoke-sub-2eb54fff-tcql2\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:33.148399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:33.148267    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2eb54fff-tcql2\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:33.148399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:33.148296    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2eb54fff-tcql2\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:33.148849 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:33.148830    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2eb54fff-tcql2\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:33.150581 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:33.150553    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2eb54fff-tcql2\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:33.160917 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:33.160899    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs7p7\" (UniqueName: \"kubernetes.io/projected/65043676-48c0-4ac1-8dba-934aea956a08-kube-api-access-gs7p7\") pod \"maas-api-revoke-sub-2eb54fff-tcql2\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:33.292529 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:33.292507    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:33.565813 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:33.565788    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"]
Nov 03 19:53:33.567765 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:53:33.567740    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65043676_48c0_4ac1_8dba_934aea956a08.slice/crio-51d13bce323a4e92c104390603f7a937fc73bb3bf1610bd6c65c531f5a13447e WatchSource:0}: Error finding container 51d13bce323a4e92c104390603f7a937fc73bb3bf1610bd6c65c531f5a13447e: Status 404 returned error can't find the container with id 51d13bce323a4e92c104390603f7a937fc73bb3bf1610bd6c65c531f5a13447e
Nov 03 19:53:34.126701 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:34.126607    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2" event={"ID":"65043676-48c0-4ac1-8dba-934aea956a08","Type":"ContainerStarted","Data":"9573d9138cdf8b8c1c4318418ab7b1293c96c87d94f72f006474b993e0a11e0d"}
Nov 03 19:53:34.126701 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:34.126671    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2" event={"ID":"65043676-48c0-4ac1-8dba-934aea956a08","Type":"ContainerStarted","Data":"51d13bce323a4e92c104390603f7a937fc73bb3bf1610bd6c65c531f5a13447e"}
Nov 03 19:53:34.146838 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:34.146791    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2" podStartSLOduration=2.146777309 podStartE2EDuration="2.146777309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:53:34.14406898 +0000 UTC m=+1615.052090917" watchObservedRunningTime="2026-11-03 19:53:34.146777309 +0000 UTC m=+1615.054799247"
Nov 03 19:53:35.134397 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:35.134365    5051 generic.go:356] "Generic (PLEG): container finished" podID="65043676-48c0-4ac1-8dba-934aea956a08" containerID="9573d9138cdf8b8c1c4318418ab7b1293c96c87d94f72f006474b993e0a11e0d" exitCode=0
Nov 03 19:53:35.134861 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:35.134444    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2" event={"ID":"65043676-48c0-4ac1-8dba-934aea956a08","Type":"ContainerDied","Data":"9573d9138cdf8b8c1c4318418ab7b1293c96c87d94f72f006474b993e0a11e0d"}
Nov 03 19:53:36.301382 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:36.301357    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:36.379886 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:36.379859    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-xks-ca\") pod \"65043676-48c0-4ac1-8dba-934aea956a08\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") "
Nov 03 19:53:36.380068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:36.379927    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-openshift-ca\") pod \"65043676-48c0-4ac1-8dba-934aea956a08\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") "
Nov 03 19:53:36.380068 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:36.380021    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65043676-48c0-4ac1-8dba-934aea956a08-kube-api-access-gs7p7\" (UniqueName: \"kubernetes.io/projected/65043676-48c0-4ac1-8dba-934aea956a08-kube-api-access-gs7p7\") pod \"65043676-48c0-4ac1-8dba-934aea956a08\" (UID: \"65043676-48c0-4ac1-8dba-934aea956a08\") "
Nov 03 19:53:36.380309 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:36.380281    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-openshift-ca" pod "65043676-48c0-4ac1-8dba-934aea956a08" (UID: "65043676-48c0-4ac1-8dba-934aea956a08"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:53:36.380421 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:36.380379    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-openshift-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:36.381970 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:36.381942    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-xks-ca" pod "65043676-48c0-4ac1-8dba-934aea956a08" (UID: "65043676-48c0-4ac1-8dba-934aea956a08"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:53:36.382081 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:36.382062    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65043676-48c0-4ac1-8dba-934aea956a08-kube-api-access-gs7p7" pod "65043676-48c0-4ac1-8dba-934aea956a08" (UID: "65043676-48c0-4ac1-8dba-934aea956a08"). InnerVolumeSpecName "kube-api-access-gs7p7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:53:36.481072 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:36.481029    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65043676-48c0-4ac1-8dba-934aea956a08-maas-api-xks-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:36.481175 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:36.481074    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs7p7\" (UniqueName: \"kubernetes.io/projected/65043676-48c0-4ac1-8dba-934aea956a08-kube-api-access-gs7p7\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:36.824896 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:36.824824    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx" podUID="e1cb7535-15c0-4151-b0b6-92cbafd2e4ac" containerName="payload-processing" containerID="cri-o://495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260" gracePeriod=25
Nov 03 19:53:37.146275 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:37.146199    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"
Nov 03 19:53:37.146421 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:37.146199    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2" event={"ID":"65043676-48c0-4ac1-8dba-934aea956a08","Type":"ContainerDied","Data":"51d13bce323a4e92c104390603f7a937fc73bb3bf1610bd6c65c531f5a13447e"}
Nov 03 19:53:37.146421 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:37.146323    5051 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51d13bce323a4e92c104390603f7a937fc73bb3bf1610bd6c65c531f5a13447e"
Nov 03 19:53:39.828424 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:39.828389    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx" podUID="e1cb7535-15c0-4151-b0b6-92cbafd2e4ac" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 19:53:41.525889 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.525851    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" containerID="cri-o://4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04" gracePeriod=45
Nov 03 19:53:41.529840 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.529806    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": dial tcp 10.132.0.77:8000: connect: connection refused"
Nov 03 19:53:41.539976 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.539948    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" containerID="cri-o://d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f" gracePeriod=45
Nov 03 19:53:41.722795 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.722696    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": dial tcp 10.132.0.69:8000: connect: connection refused"
Nov 03 19:53:41.837572 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.837548    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:53:41.840791 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.840769    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:53:41.926786 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.926755    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tls-certs\" (UniqueName: \"kubernetes.io/secret/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tls-certs\") pod \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") "
Nov 03 19:53:41.926950 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.926821    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-dshm\") pod \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") "
Nov 03 19:53:41.926950 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.926853    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-tmp-dir\") pod \"6f3db017-2c11-424f-9acb-51cfa24a92c4\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") "
Nov 03 19:53:41.926950 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.926884    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-home\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-home\") pod \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") "
Nov 03 19:53:41.926950 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.926927    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-model-cache\") pod \"6f3db017-2c11-424f-9acb-51cfa24a92c4\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") "
Nov 03 19:53:41.927197 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927023    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-home\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-home\") pod \"6f3db017-2c11-424f-9acb-51cfa24a92c4\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") "
Nov 03 19:53:41.927197 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927121    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f3db017-2c11-424f-9acb-51cfa24a92c4-kube-api-access-sztsx\" (UniqueName: \"kubernetes.io/projected/6f3db017-2c11-424f-9acb-51cfa24a92c4-kube-api-access-sztsx\") pod \"6f3db017-2c11-424f-9acb-51cfa24a92c4\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") "
Nov 03 19:53:41.927302 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927185    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-home" pod "6f3db017-2c11-424f-9acb-51cfa24a92c4" (UID: "6f3db017-2c11-424f-9acb-51cfa24a92c4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:41.927302 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927204    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-tmp-dir" pod "6f3db017-2c11-424f-9acb-51cfa24a92c4" (UID: "6f3db017-2c11-424f-9acb-51cfa24a92c4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:41.927302 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927212    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-home" pod "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" (UID: "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:41.927302 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927269    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tmp-dir\") pod \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") "
Nov 03 19:53:41.927521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927315    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-kube-api-access-88bpg\" (UniqueName: \"kubernetes.io/projected/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-kube-api-access-88bpg\") pod \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") "
Nov 03 19:53:41.927521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927416    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f3db017-2c11-424f-9acb-51cfa24a92c4-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f3db017-2c11-424f-9acb-51cfa24a92c4-tls-certs\") pod \"6f3db017-2c11-424f-9acb-51cfa24a92c4\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") "
Nov 03 19:53:41.927521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927419    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-model-cache" pod "6f3db017-2c11-424f-9acb-51cfa24a92c4" (UID: "6f3db017-2c11-424f-9acb-51cfa24a92c4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:41.927521 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927506    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-model-cache\") pod \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\" (UID: \"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1\") "
Nov 03 19:53:41.927761 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927546    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-dshm\") pod \"6f3db017-2c11-424f-9acb-51cfa24a92c4\" (UID: \"6f3db017-2c11-424f-9acb-51cfa24a92c4\") "
Nov 03 19:53:41.927761 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927571    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tmp-dir" pod "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" (UID: "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:41.927955 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.927725    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-model-cache" pod "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" (UID: "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:41.928100 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.928050    5051 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-model-cache\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:41.928100 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.928072    5051 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-tmp-dir\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:41.928100 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.928086    5051 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-home\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:41.928270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.928099    5051 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-model-cache\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:41.928270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.928114    5051 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-home\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:41.928270 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.928126    5051 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tmp-dir\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:41.929552 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.929527    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tls-certs" pod "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" (UID: "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:53:41.929678 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.929623    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-dshm" pod "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" (UID: "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:41.929880 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.929853    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-kube-api-access-88bpg" pod "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" (UID: "be6fda08-cc2c-48ed-a5bf-8dbc16810ab1"). InnerVolumeSpecName "kube-api-access-88bpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:53:41.930019 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.930001    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f3db017-2c11-424f-9acb-51cfa24a92c4-tls-certs" pod "6f3db017-2c11-424f-9acb-51cfa24a92c4" (UID: "6f3db017-2c11-424f-9acb-51cfa24a92c4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:53:41.930385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.930353    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-dshm" pod "6f3db017-2c11-424f-9acb-51cfa24a92c4" (UID: "6f3db017-2c11-424f-9acb-51cfa24a92c4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:53:41.930671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:41.930650    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f3db017-2c11-424f-9acb-51cfa24a92c4-kube-api-access-sztsx" pod "6f3db017-2c11-424f-9acb-51cfa24a92c4" (UID: "6f3db017-2c11-424f-9acb-51cfa24a92c4"). InnerVolumeSpecName "kube-api-access-sztsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:53:42.029334 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.029310    5051 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f3db017-2c11-424f-9acb-51cfa24a92c4-tls-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:42.029334 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.029334    5051 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f3db017-2c11-424f-9acb-51cfa24a92c4-dshm\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:42.029491 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.029343    5051 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-tls-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:42.029491 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.029351    5051 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-dshm\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:42.029491 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.029361    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sztsx\" (UniqueName: \"kubernetes.io/projected/6f3db017-2c11-424f-9acb-51cfa24a92c4-kube-api-access-sztsx\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:42.029491 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.029370    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88bpg\" (UniqueName: \"kubernetes.io/projected/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1-kube-api-access-88bpg\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:42.174202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.174119    5051 generic.go:356] "Generic (PLEG): container finished" podID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerID="d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f" exitCode=0
Nov 03 19:53:42.174202 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.174170    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"
Nov 03 19:53:42.174384 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.174195    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" event={"ID":"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1","Type":"ContainerDied","Data":"d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f"}
Nov 03 19:53:42.174384 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.174240    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq" event={"ID":"be6fda08-cc2c-48ed-a5bf-8dbc16810ab1","Type":"ContainerDied","Data":"6d97b709be1b557c90df710de8d1e457a4397325f35841bd65c53718304b82be"}
Nov 03 19:53:42.174384 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.174263    5051 scope.go:122] "RemoveContainer" containerID="d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f"
Nov 03 19:53:42.175592 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.175574    5051 generic.go:356] "Generic (PLEG): container finished" podID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerID="4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04" exitCode=0
Nov 03 19:53:42.175687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.175623    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"
Nov 03 19:53:42.175687 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.175666    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" event={"ID":"6f3db017-2c11-424f-9acb-51cfa24a92c4","Type":"ContainerDied","Data":"4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04"}
Nov 03 19:53:42.175778 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.175709    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz" event={"ID":"6f3db017-2c11-424f-9acb-51cfa24a92c4","Type":"ContainerDied","Data":"8cfd3f3fd467b759a7aeda4cfd333327939870c8c3da124c1a7f132724dab739"}
Nov 03 19:53:42.185229 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.185208    5051 scope.go:122] "RemoveContainer" containerID="d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f"
Nov 03 19:53:42.185574 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:53:42.185482    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f\": container with ID starting with d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f not found: ID does not exist" containerID="d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f"
Nov 03 19:53:42.185574 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.185512    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f"} err="failed to get container status \"d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f\": rpc error: code = NotFound desc = could not find container \"d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f\": container with ID starting with d6b32518b04420afd46ff2d33e4505c11feca527e1d0b9f9756f5ed34a81487f not found: ID does not exist"
Nov 03 19:53:42.185574 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.185535    5051 scope.go:122] "RemoveContainer" containerID="4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04"
Nov 03 19:53:42.193871 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.193853    5051 scope.go:122] "RemoveContainer" containerID="4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04"
Nov 03 19:53:42.194165 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:53:42.194146    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04\": container with ID starting with 4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04 not found: ID does not exist" containerID="4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04"
Nov 03 19:53:42.194238 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.194170    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04"} err="failed to get container status \"4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04\": rpc error: code = NotFound desc = could not find container \"4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04\": container with ID starting with 4a4980f2d2b76382afedea00597fa99313af12d2efaef5bd77dd00193a113d04 not found: ID does not exist"
Nov 03 19:53:42.198325 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.198301    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"]
Nov 03 19:53:42.204208 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.204178    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/premium-w4-d8a4ea-kserve-5bb684988c-vqktq"]
Nov 03 19:53:42.215150 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.215128    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"]
Nov 03 19:53:42.219429 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:42.219403    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-d8a4ea/distinct-2-w4-d8a4ea-kserve-659b474bfc-hvmxz"]
Nov 03 19:53:43.228755 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:43.228725    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" path="/var/lib/kubelet/pods/6f3db017-2c11-424f-9acb-51cfa24a92c4/volumes"
Nov 03 19:53:43.229156 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:43.229141    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" path="/var/lib/kubelet/pods/be6fda08-cc2c-48ed-a5bf-8dbc16810ab1/volumes"
Nov 03 19:53:45.302101 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:45.302062    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:53:49.828533 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:49.828495    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx" podUID="e1cb7535-15c0-4151-b0b6-92cbafd2e4ac" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 19:53:51.779024 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:51.778992    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:53:57.094394 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.094336    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:53:57.252125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.252088    5051 generic.go:356] "Generic (PLEG): container finished" podID="e1cb7535-15c0-4151-b0b6-92cbafd2e4ac" containerID="495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260" exitCode=0
Nov 03 19:53:57.252289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.252138    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"
Nov 03 19:53:57.252289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.252173    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx" event={"ID":"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac","Type":"ContainerDied","Data":"495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260"}
Nov 03 19:53:57.252289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.252233    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx" event={"ID":"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac","Type":"ContainerDied","Data":"a9074986ac38c19de429645144ac703bdff11e21ab06bd6a317eb918f67d3cfd"}
Nov 03 19:53:57.252289 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.252256    5051 scope.go:122] "RemoveContainer" containerID="495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260"
Nov 03 19:53:57.261118 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.261098    5051 scope.go:122] "RemoveContainer" containerID="495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260"
Nov 03 19:53:57.261376 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:53:57.261357    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260\": container with ID starting with 495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260 not found: ID does not exist" containerID="495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260"
Nov 03 19:53:57.261427 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.261390    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260"} err="failed to get container status \"495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260\": rpc error: code = NotFound desc = could not find container \"495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260\": container with ID starting with 495b561d100cf3641c1204284a84e6d19acc172061c63f1041f9adf9ef417260 not found: ID does not exist"
Nov 03 19:53:57.265700 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.265682    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-kube-api-access-jgq6r\" (UniqueName: \"kubernetes.io/projected/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-kube-api-access-jgq6r\") pod \"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac\" (UID: \"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac\") "
Nov 03 19:53:57.265772 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.265755    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-plugins-config-volume\") pod \"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac\" (UID: \"e1cb7535-15c0-4151-b0b6-92cbafd2e4ac\") "
Nov 03 19:53:57.266125 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.266105    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-plugins-config-volume" pod "e1cb7535-15c0-4151-b0b6-92cbafd2e4ac" (UID: "e1cb7535-15c0-4151-b0b6-92cbafd2e4ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:53:57.267717 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.267693    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-kube-api-access-jgq6r" pod "e1cb7535-15c0-4151-b0b6-92cbafd2e4ac" (UID: "e1cb7535-15c0-4151-b0b6-92cbafd2e4ac"). InnerVolumeSpecName "kube-api-access-jgq6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:53:57.367133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.367077    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgq6r\" (UniqueName: \"kubernetes.io/projected/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-kube-api-access-jgq6r\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:57.367133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.367100    5051 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac-plugins-config-volume\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:53:57.575163 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.575133    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"]
Nov 03 19:53:57.579820 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:57.579795    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a4ea-6fbdbfbbdf-cmqjx"]
Nov 03 19:53:59.227339 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:53:59.227309    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1cb7535-15c0-4151-b0b6-92cbafd2e4ac" path="/var/lib/kubelet/pods/e1cb7535-15c0-4151-b0b6-92cbafd2e4ac/volumes"
Nov 03 19:54:02.769582 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:54:02.769546    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:54:32.766462 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:54:32.766363    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:55:02.767412 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:55:02.767376    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:55:08.397459 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:55:08.397418    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:55:11.065185 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:55:11.065129    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:55:11.065847 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:55:11.065796    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:55:26.798626 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:55:26.798584    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:55:31.309960 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:55:31.309921    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:55:32.766850 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:55:32.766810    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:55:45.711813 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:55:45.711770    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:55:50.404065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:55:50.400416    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qtcsx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:55:50.404065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:55:50.400480    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx" podUID="159b05c6-8444-4ed2-95d7-7d07dd9d8b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:55:50.404065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:55:50.402118    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xtsbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:55:50.404065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:55:50.402167    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" podUID="9b1113b4-950d-4cf7-a856-1290aa4a1934" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:55:58.686582 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:55:58.686546    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:56:02.767014 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:56:02.766973    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:56:23.601405 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:56:23.601370    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:56:26.064065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:56:26.063123    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-c684f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:56:26.064065 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:56:26.063199    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-c684f" podUID="b25cf9ad-db43-42cb-8fc8-883c249e8796" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:56:32.767418 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:56:32.767381    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:56:33.754520 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:56:33.754488    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:56:38.036766 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:56:38.036732    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:56:40.485577 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:56:40.485550    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:56:40.613738 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:56:40.613713    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 19:56:54.789982 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:56:54.789945    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:56:59.085064 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:56:59.084357    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:57:02.767206 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:57:02.767159    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:57:12.791261 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:12.791223    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:57:16.925034 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:16.924990    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:57:31.401178 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:31.401085    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:57:32.767444 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:57:32.767408    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:57:35.625858 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:35.625821    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:57:38.061523 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.061490    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"]
Nov 03 19:57:38.062024 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.061772    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x" podUID="36d8cf52-dffc-4912-9d2c-d4b55e57a769" containerName="discovery" containerID="cri-o://e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4" gracePeriod=30
Nov 03 19:57:38.328000 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.327915    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:57:38.345532 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.345505    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36d8cf52-dffc-4912-9d2c-d4b55e57a769-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36d8cf52-dffc-4912-9d2c-d4b55e57a769-local-certs\") pod \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") "
Nov 03 19:57:38.345681 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.345543    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-cacerts\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-cacerts\") pod \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") "
Nov 03 19:57:38.345681 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.345574    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-ca-configmap\") pod \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") "
Nov 03 19:57:38.345681 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.345600    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-kubeconfig\") pod \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") "
Nov 03 19:57:38.345681 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.345654    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-token\" (UniqueName: \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-token\") pod \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") "
Nov 03 19:57:38.345906 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.345816    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-kube-api-access-x9tlt\" (UniqueName: \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-kube-api-access-x9tlt\") pod \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") "
Nov 03 19:57:38.346034 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.346008    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-ca-configmap" pod "36d8cf52-dffc-4912-9d2c-d4b55e57a769" (UID: "36d8cf52-dffc-4912-9d2c-d4b55e57a769"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 19:57:38.346133 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.346095    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-dns-cert\") pod \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\" (UID: \"36d8cf52-dffc-4912-9d2c-d4b55e57a769\") "
Nov 03 19:57:38.346608 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.346581    5051 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-ca-configmap\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:57:38.347939 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.347904    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36d8cf52-dffc-4912-9d2c-d4b55e57a769-local-certs" pod "36d8cf52-dffc-4912-9d2c-d4b55e57a769" (UID: "36d8cf52-dffc-4912-9d2c-d4b55e57a769"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 19:57:38.348472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.348434    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-kubeconfig" pod "36d8cf52-dffc-4912-9d2c-d4b55e57a769" (UID: "36d8cf52-dffc-4912-9d2c-d4b55e57a769"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:57:38.348472 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.348455    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-token" pod "36d8cf52-dffc-4912-9d2c-d4b55e57a769" (UID: "36d8cf52-dffc-4912-9d2c-d4b55e57a769"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:57:38.348617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.348510    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-cacerts" pod "36d8cf52-dffc-4912-9d2c-d4b55e57a769" (UID: "36d8cf52-dffc-4912-9d2c-d4b55e57a769"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:57:38.348678 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.348625    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-dns-cert" pod "36d8cf52-dffc-4912-9d2c-d4b55e57a769" (UID: "36d8cf52-dffc-4912-9d2c-d4b55e57a769"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 19:57:38.348820 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.348799    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-kube-api-access-x9tlt" pod "36d8cf52-dffc-4912-9d2c-d4b55e57a769" (UID: "36d8cf52-dffc-4912-9d2c-d4b55e57a769"). InnerVolumeSpecName "kube-api-access-x9tlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 19:57:38.447854 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.447825    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:57:38.447854 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.447853    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9tlt\" (UniqueName: \"kubernetes.io/projected/36d8cf52-dffc-4912-9d2c-d4b55e57a769-kube-api-access-x9tlt\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:57:38.448061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.447864    5051 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-csr-dns-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:57:38.448061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.447874    5051 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36d8cf52-dffc-4912-9d2c-d4b55e57a769-local-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:57:38.448061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.447882    5051 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-cacerts\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:57:38.448061 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.447891    5051 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36d8cf52-dffc-4912-9d2c-d4b55e57a769-istio-kubeconfig\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 19:57:38.638182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.638104    5051 generic.go:356] "Generic (PLEG): container finished" podID="36d8cf52-dffc-4912-9d2c-d4b55e57a769" containerID="e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4" exitCode=0
Nov 03 19:57:38.638182 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.638155    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"
Nov 03 19:57:38.638379 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.638190    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x" event={"ID":"36d8cf52-dffc-4912-9d2c-d4b55e57a769","Type":"ContainerDied","Data":"e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4"}
Nov 03 19:57:38.638379 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.638241    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x" event={"ID":"36d8cf52-dffc-4912-9d2c-d4b55e57a769","Type":"ContainerDied","Data":"af37fd699720ab2669deef0d1bdda8f5609a2e1aa9bbbfea18374474687dee27"}
Nov 03 19:57:38.638379 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.638260    5051 scope.go:122] "RemoveContainer" containerID="e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4"
Nov 03 19:57:38.649001 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.648983    5051 scope.go:122] "RemoveContainer" containerID="e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4"
Nov 03 19:57:38.649307 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:57:38.649286    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4\": container with ID starting with e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4 not found: ID does not exist" containerID="e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4"
Nov 03 19:57:38.649385 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.649315    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4"} err="failed to get container status \"e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4\": rpc error: code = NotFound desc = could not find container \"e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4\": container with ID starting with e5c8c237ede377eb37d9b3e165ba86845e371c82126105cb4b90f35abc7badf4 not found: ID does not exist"
Nov 03 19:57:38.662898 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.662873    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"]
Nov 03 19:57:38.669466 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:38.669445    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxs8x"]
Nov 03 19:57:39.028077 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:39.028032    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"]
Nov 03 19:57:39.033155 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:39.033124    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d72923e3-5mbv6"]
Nov 03 19:57:39.229105 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:39.229064    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36d8cf52-dffc-4912-9d2c-d4b55e57a769" path="/var/lib/kubelet/pods/36d8cf52-dffc-4912-9d2c-d4b55e57a769/volumes"
Nov 03 19:57:39.229669 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:39.229651    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="395bf147-d73d-4c0d-9693-93b6b8e15731" path="/var/lib/kubelet/pods/395bf147-d73d-4c0d-9693-93b6b8e15731/volumes"
Nov 03 19:57:49.263428 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:49.263392    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:57:50.022280 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:50.022236    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"]
Nov 03 19:57:50.026818 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:50.026779    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66361ab1-lkws6"]
Nov 03 19:57:51.228739 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:51.228705    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30822646-6bb1-48e1-8a7f-07cab19a7b0d" path="/var/lib/kubelet/pods/30822646-6bb1-48e1-8a7f-07cab19a7b0d/volumes"
Nov 03 19:57:59.875248 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:57:59.875212    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:58:02.792463 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:58:02.792421    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:58:05.113067 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:58:05.112276    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:58:05.372281 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:58:05.372185    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xtsbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 19:58:05.372554 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:58:05.372514    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" podUID="9b1113b4-950d-4cf7-a856-1290aa4a1934" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 19:58:28.358118 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:58:28.358086    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:58:32.766743 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:58:32.766700    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:58:37.024860 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:58:37.024808    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"]
Nov 03 19:58:37.027988 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:58:37.027960    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2eb54fff-tcql2"]
Nov 03 19:58:37.227797 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:58:37.227761    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65043676-48c0-4ac1-8dba-934aea956a08" path="/var/lib/kubelet/pods/65043676-48c0-4ac1-8dba-934aea956a08/volumes"
Nov 03 19:58:42.751355 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:58:42.751328    5051 scope.go:122] "RemoveContainer" containerID="6923c0fa2fa7d4a1da682e4b18d6b8e0883321e44cbd8c483ab226a7fec7f0d5"
Nov 03 19:58:42.759874 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:58:42.759851    5051 scope.go:122] "RemoveContainer" containerID="6139637d206ee4ea1f1df03c079b9163ad8c552f55dac3e1f397bb96eae0f31d"
Nov 03 19:58:46.705356 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:58:46.705321    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:58:56.896782 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:58:56.896697    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:59:02.766606 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:59:02.766567    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:59:17.608848 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.608813    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"]
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609301    5051 cpu_manager.go:414] "containerMap: removing container" podUID="36d8cf52-dffc-4912-9d2c-d4b55e57a769" containerName="discovery"
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609314    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36d8cf52-dffc-4912-9d2c-d4b55e57a769" containerName="discovery"
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609325    5051 cpu_manager.go:414] "containerMap: removing container" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main"
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609330    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main"
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609343    5051 cpu_manager.go:414] "containerMap: removing container" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main"
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609348    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main"
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609362    5051 cpu_manager.go:414] "containerMap: removing container" podUID="e1cb7535-15c0-4151-b0b6-92cbafd2e4ac" containerName="payload-processing"
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609369    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1cb7535-15c0-4151-b0b6-92cbafd2e4ac" containerName="payload-processing"
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609382    5051 cpu_manager.go:414] "containerMap: removing container" podUID="65043676-48c0-4ac1-8dba-934aea956a08" containerName="revoke-keys"
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609388    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65043676-48c0-4ac1-8dba-934aea956a08" containerName="revoke-keys"
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609401    5051 cpu_manager.go:414] "containerMap: removing container" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy"
Nov 03 19:59:17.609447 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609406    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="701528a6-3102-4145-8c38-c31e6f80cd9f" containerName="istio-proxy"
Nov 03 19:59:17.610121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609477    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="332ebb89-3bbd-47f3-a47e-33138a5bc16e" containerName="istio-proxy"
Nov 03 19:59:17.610121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609487    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1cb7535-15c0-4151-b0b6-92cbafd2e4ac" containerName="payload-processing"
Nov 03 19:59:17.610121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609494    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a258474-8531-448f-8e4a-b7e4b39728fc" containerName="istio-proxy"
Nov 03 19:59:17.610121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609504    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="65043676-48c0-4ac1-8dba-934aea956a08" containerName="revoke-keys"
Nov 03 19:59:17.610121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609519    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="36d8cf52-dffc-4912-9d2c-d4b55e57a769" containerName="discovery"
Nov 03 19:59:17.610121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609530    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="be6fda08-cc2c-48ed-a5bf-8dbc16810ab1" containerName="main"
Nov 03 19:59:17.610121 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.609540    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f3db017-2c11-424f-9acb-51cfa24a92c4" containerName="main"
Nov 03 19:59:17.612877 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.612854    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.617346 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.617326    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-458393-openshift-default-dockercfg-m2lh8\""
Nov 03 19:59:17.636582 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.636554    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"]
Nov 03 19:59:17.684673 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.684641    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-envoy\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.684805 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.684679    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istiod-ca-cert\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.684874 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.684840    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzsxz\" (UniqueName: \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-kube-api-access-rzsxz\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.684953 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.684937    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-socket\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.685014 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.684970    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-token\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.685014 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.685008    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-podinfo\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.685165 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.685143    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-credential-socket\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.685224 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.685188    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-data\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.685291 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.685277    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-certs\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.786671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.786637    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-socket\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.786671 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.786674    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-token\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.786897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.786698    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-podinfo\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.786897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.786737    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-credential-socket\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.786897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.786757    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-data\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.786897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.786786    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-certs\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.786897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.786816    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-envoy\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.786897 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.786846    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istiod-ca-cert\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.787269 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.786907    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzsxz\" (UniqueName: \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-kube-api-access-rzsxz\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.787269 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.787130    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-socket\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.787269 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.787204    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-credential-socket\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.787408 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.787374    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-data\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.787466 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.787430    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-certs\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.787718 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.787701    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istiod-ca-cert\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.789084 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.789060    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-envoy\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.789423 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.789403    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-podinfo\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.799333 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.799311    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-token\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.799993 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.799976    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzsxz\" (UniqueName: \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-kube-api-access-rzsxz\") pod \"e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") " pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:17.928344 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:17.928266    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:18.205654 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:18.205629    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"]
Nov 03 19:59:18.206876 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:59:18.206849    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9fb361_a352_4b4d_95b8_3d6a1628a9f1.slice/crio-bf3bd75f66d28f62c24e802e17b101f540627511ddd23242984b93142f69e88d WatchSource:0}: Error finding container bf3bd75f66d28f62c24e802e17b101f540627511ddd23242984b93142f69e88d: Status 404 returned error can't find the container with id bf3bd75f66d28f62c24e802e17b101f540627511ddd23242984b93142f69e88d
Nov 03 19:59:18.209011 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:18.208979    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:59:18.209124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:18.209054    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:59:18.209124 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:18.209092    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:59:18.269288 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:18.269262    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" event={"ID":"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1","Type":"ContainerStarted","Data":"bf3bd75f66d28f62c24e802e17b101f540627511ddd23242984b93142f69e88d"}
Nov 03 19:59:19.275487 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:19.275453    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" event={"ID":"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1","Type":"ContainerStarted","Data":"c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf"}
Nov 03 19:59:19.301546 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:19.301496    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" podStartSLOduration=2.301483691 podStartE2EDuration="2.301483691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:59:19.299778975 +0000 UTC m=+1960.207800925" watchObservedRunningTime="2026-11-03 19:59:19.301483691 +0000 UTC m=+1960.209505630"
Nov 03 19:59:19.928910 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:19.928869    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:19.934162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:19.934137    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:20.280399 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:20.280355    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:20.281162 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:20.281142    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:22.849066 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:22.849005    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"]
Nov 03 19:59:22.853062 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:22.853028    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:22.860251 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:22.860223    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-458393-serving-cert\""
Nov 03 19:59:22.876184 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:22.876147    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"]
Nov 03 19:59:23.045596 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.045556    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxsvb\" (UniqueName: \"kubernetes.io/projected/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-kube-api-access-sxsvb\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.045776 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.045629    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-metrics-certs\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.045776 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.045665    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.045776 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.045707    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.153252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.147061    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxsvb\" (UniqueName: \"kubernetes.io/projected/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-kube-api-access-sxsvb\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.153252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.147148    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-metrics-certs\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.153252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.147564    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.153252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.147623    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.153252 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:59:23.147797    5051 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-serving-cert: secret "maas-api-e2e-worker-single-main-458393-serving-cert" not found
Nov 03 19:59:23.153252 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:59:23.147861    5051 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-maas-api-tls podName:c9afbd67-ec51-4a67-82b7-cb99cd0795f9 nodeName:}" failed. No retries permitted until 2026-11-03 19:59:23.647838705 +0000 UTC m=+1964.555860627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-maas-api-tls") pod "maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv" (UID: "c9afbd67-ec51-4a67-82b7-cb99cd0795f9") : secret "maas-api-e2e-worker-single-main-458393-serving-cert" not found
Nov 03 19:59:23.153252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.148527    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.153252 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.150455    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-metrics-certs\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.186132 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.186090    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxsvb\" (UniqueName: \"kubernetes.io/projected/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-kube-api-access-sxsvb\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.653228 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.653197    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.656109 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.656075    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:23.769468 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:23.769434    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:24.050877 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:24.050846    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"]
Nov 03 19:59:24.052351 ip-10-0-143-212 kubenswrapper[5051]: W1103 19:59:24.052326    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9afbd67_ec51_4a67_82b7_cb99cd0795f9.slice/crio-9a8ac59370e818b197c2f99364da4fe6d5c5a1baae648e92080a83bff7afd236 WatchSource:0}: Error finding container 9a8ac59370e818b197c2f99364da4fe6d5c5a1baae648e92080a83bff7afd236: Status 404 returned error can't find the container with id 9a8ac59370e818b197c2f99364da4fe6d5c5a1baae648e92080a83bff7afd236
Nov 03 19:59:24.053741 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:24.053721    5051 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 19:59:24.301219 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:24.301148    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv" event={"ID":"c9afbd67-ec51-4a67-82b7-cb99cd0795f9","Type":"ContainerStarted","Data":"9a8ac59370e818b197c2f99364da4fe6d5c5a1baae648e92080a83bff7afd236"}
Nov 03 19:59:25.308154 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:25.308120    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv" event={"ID":"c9afbd67-ec51-4a67-82b7-cb99cd0795f9","Type":"ContainerStarted","Data":"bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618"}
Nov 03 19:59:25.308566 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:25.308246    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:25.331697 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:25.331645    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv" podStartSLOduration=2.844167936 podStartE2EDuration="3.331629194s" totalImagesPullingTime="487.461258ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 19:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 19:59:25.330558397 +0000 UTC m=+1966.238580336" watchObservedRunningTime="2026-11-03 19:59:25.331629194 +0000 UTC m=+1966.239651135"
Nov 03 19:59:25.716099 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:25.716061    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:59:31.319139 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:31.319107    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 19:59:32.767901 ip-10-0-143-212 kubenswrapper[5051]: E1103 19:59:32.767859    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 19:59:35.281860 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:35.281821    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 03 19:59:35.282292 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:35.281893    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 03 19:59:35.362272 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:35.362204    5051 generic.go:356] "Generic (PLEG): container finished" podID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerID="c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf" exitCode=0
Nov 03 19:59:35.362272 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:35.362266    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" event={"ID":"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1","Type":"ContainerDied","Data":"c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf"}
Nov 03 19:59:35.362611 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:35.362587    5051 scope.go:122] "RemoveContainer" containerID="c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf"
Nov 03 19:59:35.363775 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:35.363734    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:59:35.363888 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:35.363833    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:59:35.363888 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:35.363880    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 19:59:36.372336 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:36.372302    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" event={"ID":"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1","Type":"ContainerStarted","Data":"2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce"}
Nov 03 19:59:36.929095 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:36.929055    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:36.929295 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:36.929222    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 03 19:59:36.929295 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:36.929267    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 03 19:59:37.929390 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:37.929351    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 03 19:59:37.929804 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:37.929414    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 03 19:59:38.928632 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:38.928603    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 03 19:59:38.928796 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:38.928652    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 03 19:59:39.929539 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:39.929503    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 03 19:59:39.929928 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:39.929565    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 03 19:59:39.993586 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:39.993555    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 19:59:40.928869 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:40.928831    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 03 19:59:40.929060 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:40.928897    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 03 19:59:41.932946 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:41.932917    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:41.933391 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:41.933292    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:41.934035 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:41.934016    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 19:59:42.783967 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:42.783934    5051 scope.go:122] "RemoveContainer" containerID="9573d9138cdf8b8c1c4318418ab7b1293c96c87d94f72f006474b993e0a11e0d"
Nov 03 19:59:42.792617 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:42.792595    5051 scope.go:122] "RemoveContainer" containerID="33c1279e4820b8192d62172b7c8e0abaaec788bcf5b96728bef6e69e3961db2b"
Nov 03 19:59:56.489294 ip-10-0-143-212 kubenswrapper[5051]: I1103 19:59:56.489262    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:00:02.766754 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:00:02.766707    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:00:20.613207 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:20.613169    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:00:29.949752 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:29.949717    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"]
Nov 03 20:00:29.950336 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:29.949981    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv" podUID="c9afbd67-ec51-4a67-82b7-cb99cd0795f9" containerName="maas-api" containerID="cri-o://bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618" gracePeriod=30
Nov 03 20:00:30.226837 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.226804    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 20:00:30.333163 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.333132    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-maas-api-tls\") pod \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") "
Nov 03 20:00:30.333353 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.333255    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-kube-api-access-sxsvb\" (UniqueName: \"kubernetes.io/projected/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-kube-api-access-sxsvb\") pod \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") "
Nov 03 20:00:30.333353 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.333297    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-metrics-certs\") pod \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") "
Nov 03 20:00:30.333542 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.333445    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-openshift-service-ca\") pod \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\" (UID: \"c9afbd67-ec51-4a67-82b7-cb99cd0795f9\") "
Nov 03 20:00:30.333695 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.333666    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-openshift-service-ca" pod "c9afbd67-ec51-4a67-82b7-cb99cd0795f9" (UID: "c9afbd67-ec51-4a67-82b7-cb99cd0795f9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:00:30.333933 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.333914    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:30.335476 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.335449    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-maas-api-tls" pod "c9afbd67-ec51-4a67-82b7-cb99cd0795f9" (UID: "c9afbd67-ec51-4a67-82b7-cb99cd0795f9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:00:30.335476 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.335460    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-kube-api-access-sxsvb" pod "c9afbd67-ec51-4a67-82b7-cb99cd0795f9" (UID: "c9afbd67-ec51-4a67-82b7-cb99cd0795f9"). InnerVolumeSpecName "kube-api-access-sxsvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:00:30.335598 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.335472    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-metrics-certs" pod "c9afbd67-ec51-4a67-82b7-cb99cd0795f9" (UID: "c9afbd67-ec51-4a67-82b7-cb99cd0795f9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:00:30.434997 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.434964    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:30.434997 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.434993    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxsvb\" (UniqueName: \"kubernetes.io/projected/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-kube-api-access-sxsvb\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:30.434997 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.435003    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9afbd67-ec51-4a67-82b7-cb99cd0795f9-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:30.663359 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.663319    5051 generic.go:356] "Generic (PLEG): container finished" podID="c9afbd67-ec51-4a67-82b7-cb99cd0795f9" containerID="bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618" exitCode=0
Nov 03 20:00:30.663534 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.663370    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"
Nov 03 20:00:30.663534 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.663403    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv" event={"ID":"c9afbd67-ec51-4a67-82b7-cb99cd0795f9","Type":"ContainerDied","Data":"bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618"}
Nov 03 20:00:30.663534 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.663455    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv" event={"ID":"c9afbd67-ec51-4a67-82b7-cb99cd0795f9","Type":"ContainerDied","Data":"9a8ac59370e818b197c2f99364da4fe6d5c5a1baae648e92080a83bff7afd236"}
Nov 03 20:00:30.663534 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.663479    5051 scope.go:122] "RemoveContainer" containerID="bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618"
Nov 03 20:00:30.673036 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.673013    5051 scope.go:122] "RemoveContainer" containerID="bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618"
Nov 03 20:00:30.673351 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:00:30.673333    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618\": container with ID starting with bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618 not found: ID does not exist" containerID="bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618"
Nov 03 20:00:30.673406 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.673362    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618"} err="failed to get container status \"bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618\": rpc error: code = NotFound desc = could not find container \"bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618\": container with ID starting with bba4f599ac9793280b4fe7ce4af7683564a448c3b7c7cd2e76249dd72d6f3618 not found: ID does not exist"
Nov 03 20:00:30.687882 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.687860    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"]
Nov 03 20:00:30.691773 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:30.691752    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-458393-99cdf9f4d-qdwlv"]
Nov 03 20:00:31.228661 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:31.228626    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9afbd67-ec51-4a67-82b7-cb99cd0795f9" path="/var/lib/kubelet/pods/c9afbd67-ec51-4a67-82b7-cb99cd0795f9/volumes"
Nov 03 20:00:31.878328 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:31.878294    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:00:32.765864 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:00:32.765829    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:00:37.246564 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:37.246523    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"]
Nov 03 20:00:37.247129 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:37.246827    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy" containerID="cri-o://2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce" gracePeriod=30
Nov 03 20:00:41.936519 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:41.936479    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 20:00:41.936903 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:41.936550    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 20:00:42.498915 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.498887    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 20:00:42.657966 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.657876    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-socket\") pod \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") "
Nov 03 20:00:42.657966 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.657922    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-podinfo\") pod \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") "
Nov 03 20:00:42.658244 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.657977    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-certs\") pod \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") "
Nov 03 20:00:42.658244 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658137    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-kube-api-access-rzsxz\" (UniqueName: \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-kube-api-access-rzsxz\") pod \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") "
Nov 03 20:00:42.658244 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658151    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-socket" pod "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" (UID: "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:00:42.658244 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658173    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istiod-ca-cert\") pod \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") "
Nov 03 20:00:42.658244 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658206    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-envoy\") pod \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") "
Nov 03 20:00:42.658244 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658244    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-credential-socket\") pod \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") "
Nov 03 20:00:42.658573 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658278    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-data\") pod \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") "
Nov 03 20:00:42.658573 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658291    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-certs" pod "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" (UID: "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:00:42.658573 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658377    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-token\" (UniqueName: \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-token\") pod \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\" (UID: \"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1\") "
Nov 03 20:00:42.658573 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658540    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-data" pod "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" (UID: "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:00:42.658812 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658575    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istiod-ca-cert" pod "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" (UID: "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:00:42.658812 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658579    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-credential-socket" pod "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" (UID: "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:00:42.658812 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658802    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:42.658991 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658823    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:42.658991 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658841    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:42.658991 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658857    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:42.658991 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.658872    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:42.660293 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.660262    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-podinfo" pod "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" (UID: "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 20:00:42.660293 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.660279    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-kube-api-access-rzsxz" pod "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" (UID: "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1"). InnerVolumeSpecName "kube-api-access-rzsxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:00:42.660450 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.660298    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-envoy" pod "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" (UID: "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:00:42.660532 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.660517    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-token" pod "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" (UID: "0c9fb361-a352-4b4d-95b8-3d6a1628a9f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:00:42.729578 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.729546    5051 generic.go:356] "Generic (PLEG): container finished" podID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerID="2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce" exitCode=0
Nov 03 20:00:42.729739 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.729600    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"
Nov 03 20:00:42.729739 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.729625    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" event={"ID":"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1","Type":"ContainerDied","Data":"2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce"}
Nov 03 20:00:42.729739 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.729675    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj" event={"ID":"0c9fb361-a352-4b4d-95b8-3d6a1628a9f1","Type":"ContainerDied","Data":"bf3bd75f66d28f62c24e802e17b101f540627511ddd23242984b93142f69e88d"}
Nov 03 20:00:42.729739 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.729698    5051 scope.go:122] "RemoveContainer" containerID="2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce"
Nov 03 20:00:42.739612 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.739595    5051 scope.go:122] "RemoveContainer" containerID="c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf"
Nov 03 20:00:42.748361 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.748341    5051 scope.go:122] "RemoveContainer" containerID="2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce"
Nov 03 20:00:42.748639 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:00:42.748618    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce\": container with ID starting with 2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce not found: ID does not exist" containerID="2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce"
Nov 03 20:00:42.748732 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.748650    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce"} err="failed to get container status \"2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce\": rpc error: code = NotFound desc = could not find container \"2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce\": container with ID starting with 2bc6c1d5efaa831420e261fea6b076155bea6aa6aa7c1d5bdd84b00a69ab5dce not found: ID does not exist"
Nov 03 20:00:42.748732 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.748672    5051 scope.go:122] "RemoveContainer" containerID="c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf"
Nov 03 20:00:42.748934 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:00:42.748917    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf\": container with ID starting with c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf not found: ID does not exist" containerID="c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf"
Nov 03 20:00:42.748973 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.748941    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf"} err="failed to get container status \"c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf\": rpc error: code = NotFound desc = could not find container \"c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf\": container with ID starting with c78731a83021e99dfb4cfaaa8b18b695268f41ddfd239b81d24d1a6eddc0ccdf not found: ID does not exist"
Nov 03 20:00:42.753761 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.753742    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"]
Nov 03 20:00:42.757947 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.757924    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-458393-openshift-default-6b48c4d6f75b4lj"]
Nov 03 20:00:42.759372 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.759351    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:42.759449 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.759374    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzsxz\" (UniqueName: \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-kube-api-access-rzsxz\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:42.759449 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.759385    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:42.759449 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:42.759393    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:00:43.228232 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:00:43.228198    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" path="/var/lib/kubelet/pods/0c9fb361-a352-4b4d-95b8-3d6a1628a9f1/volumes"
Nov 03 20:01:02.766983 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:01:02.766936    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:01:32.766784 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:01:32.766724    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:01:40.520477 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:01:40.520447    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 20:01:40.643932 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:01:40.643905    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 20:02:02.766956 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:02:02.766916    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:02:15.513297 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:15.513262    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:02:30.999296 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:30.999261    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:02:32.769964 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:02:32.769924    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:02:35.406066 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:35.405166    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-xtsbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 20:02:35.406066 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:35.405264    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-xtsbv" podUID="9b1113b4-950d-4cf7-a856-1290aa4a1934" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 20:02:35.415217 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:35.415182    5051 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-qtcsx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 20:02:35.415463 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:35.415435    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-qtcsx" podUID="159b05c6-8444-4ed2-95d7-7d07dd9d8b34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 20:02:36.442377 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:36.442331    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:02:50.085957 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:50.085919    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:02:51.107610 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.107568    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"]
Nov 03 20:02:51.108286 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.108263    5051 cpu_manager.go:414] "containerMap: removing container" podUID="c9afbd67-ec51-4a67-82b7-cb99cd0795f9" containerName="maas-api"
Nov 03 20:02:51.108340 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.108289    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9afbd67-ec51-4a67-82b7-cb99cd0795f9" containerName="maas-api"
Nov 03 20:02:51.108340 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.108311    5051 cpu_manager.go:414] "containerMap: removing container" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy"
Nov 03 20:02:51.108340 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.108319    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy"
Nov 03 20:02:51.108444 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.108349    5051 cpu_manager.go:414] "containerMap: removing container" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy"
Nov 03 20:02:51.108444 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.108357    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy"
Nov 03 20:02:51.108507 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.108464    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9afbd67-ec51-4a67-82b7-cb99cd0795f9" containerName="maas-api"
Nov 03 20:02:51.108507 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.108478    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy"
Nov 03 20:02:51.108507 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.108491    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c9fb361-a352-4b4d-95b8-3d6a1628a9f1" containerName="istio-proxy"
Nov 03 20:02:51.110766 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.110747    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.126760 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.126739    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-fcdf83-openshift-default-dockercfg-fsl4t\""
Nov 03 20:02:51.138383 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.138347    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"]
Nov 03 20:02:51.181562 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.181531    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-data\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.181699 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.181577    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfxm8\" (UniqueName: \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-kube-api-access-sfxm8\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.181699 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.181655    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-podinfo\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.181779 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.181703    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9089be58-dbe7-4c22-bb32-7006c5f86423-istiod-ca-cert\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.181779 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.181726    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-socket\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.181845 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.181797    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-certs\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.181845 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.181834    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-credential-socket\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.181912 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.181879    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-envoy\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.181997 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.181983    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-token\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.282950 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.282915    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfxm8\" (UniqueName: \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-kube-api-access-sfxm8\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283159 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.282968    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-podinfo\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283159 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.283014    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9089be58-dbe7-4c22-bb32-7006c5f86423-istiod-ca-cert\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283159 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.283059    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-socket\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283354 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.283194    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-certs\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283354 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.283249    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-credential-socket\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283354 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.283324    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-envoy\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283510 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.283380    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-token\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283510 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.283441    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-data\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283630 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.283565    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-certs\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283689 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.283660    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-socket\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283814 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.283659    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-credential-socket\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.283976 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.283953    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9089be58-dbe7-4c22-bb32-7006c5f86423-istiod-ca-cert\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.284145 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.284125    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-data\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.285727 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.285702    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-envoy\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.286034 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.286014    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-podinfo\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.297233 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.297184    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-token\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.302507 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.302480    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfxm8\" (UniqueName: \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-kube-api-access-sfxm8\") pod \"e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") " pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.423213 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.423127    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:51.702633 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.702606    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"]
Nov 03 20:02:51.704455 ip-10-0-143-212 kubenswrapper[5051]: W1103 20:02:51.704418    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9089be58_dbe7_4c22_bb32_7006c5f86423.slice/crio-73e0a28c6f0e798a0516929558cba98e6363d80b5562b1750c7eabc20ee8c610 WatchSource:0}: Error finding container 73e0a28c6f0e798a0516929558cba98e6363d80b5562b1750c7eabc20ee8c610: Status 404 returned error can't find the container with id 73e0a28c6f0e798a0516929558cba98e6363d80b5562b1750c7eabc20ee8c610
Nov 03 20:02:51.706564 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.706534    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:02:51.706645 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.706604    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:02:51.706645 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:51.706632    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:02:52.400393 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:52.400352    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" event={"ID":"9089be58-dbe7-4c22-bb32-7006c5f86423","Type":"ContainerStarted","Data":"2cb51d86a176161a974bbae847b49fd46a3438e447501b811a5727dbc7994565"}
Nov 03 20:02:52.400790 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:52.400411    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" event={"ID":"9089be58-dbe7-4c22-bb32-7006c5f86423","Type":"ContainerStarted","Data":"73e0a28c6f0e798a0516929558cba98e6363d80b5562b1750c7eabc20ee8c610"}
Nov 03 20:02:52.423775 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:52.423751    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:52.429392 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:52.429366    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:52.430606 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:52.430561    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" podStartSLOduration=1.430549021 podStartE2EDuration="1.430549021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:02:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:02:52.427738619 +0000 UTC m=+2173.335760573" watchObservedRunningTime="2026-11-03 20:02:52.430549021 +0000 UTC m=+2173.338571004"
Nov 03 20:02:53.405952 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:53.405919    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:53.406735 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:53.406716    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:02:55.999304 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:55.999271    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"]
Nov 03 20:02:56.002202 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.002177    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.009418 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.009394    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-fcdf83-serving-cert\""
Nov 03 20:02:56.016079 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.015741    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"]
Nov 03 20:02:56.131221 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.131181    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.131435 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.131248    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-metrics-certs\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.131435 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.131351    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkhjh\" (UniqueName: \"kubernetes.io/projected/3b156dad-d0ed-4ed1-89ea-682467e1ca39-kube-api-access-vkhjh\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.131435 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.131413    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b156dad-d0ed-4ed1-89ea-682467e1ca39-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.232683 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.232645    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b156dad-d0ed-4ed1-89ea-682467e1ca39-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.232850 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.232711    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.232850 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.232764    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-metrics-certs\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.232850 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.232799    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkhjh\" (UniqueName: \"kubernetes.io/projected/3b156dad-d0ed-4ed1-89ea-682467e1ca39-kube-api-access-vkhjh\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.233454 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.233424    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b156dad-d0ed-4ed1-89ea-682467e1ca39-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.235455 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.235425    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.235577 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.235456    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-metrics-certs\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.255789 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.255734    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkhjh\" (UniqueName: \"kubernetes.io/projected/3b156dad-d0ed-4ed1-89ea-682467e1ca39-kube-api-access-vkhjh\") pod \"maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.316641 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.316610    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:56.623668 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:56.623584    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"]
Nov 03 20:02:57.429001 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:57.428922    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5" event={"ID":"3b156dad-d0ed-4ed1-89ea-682467e1ca39","Type":"ContainerStarted","Data":"78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d"}
Nov 03 20:02:57.429001 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:57.428977    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5" event={"ID":"3b156dad-d0ed-4ed1-89ea-682467e1ca39","Type":"ContainerStarted","Data":"9b2fcec124525fb588fb04f85b2dfd519659359cc595a5a3ccefbde44878a425"}
Nov 03 20:02:57.429421 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:57.429019    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:02:57.452281 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:57.452238    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5" podStartSLOduration=2.029669937 podStartE2EDuration="2.452226018s" totalImagesPullingTime="422.556081ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:02:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:02:57.451401379 +0000 UTC m=+2178.359423315" watchObservedRunningTime="2026-11-03 20:02:57.452226018 +0000 UTC m=+2178.360247958"
Nov 03 20:02:59.114573 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:02:59.114539    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:03:02.790281 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:03:02.790246    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:03:03.462064 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:03.440159    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:03:08.893715 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:08.893600    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": read tcp 10.132.0.2:50714->10.132.0.146:15021: read: connection reset by peer" start-of-body=
Nov 03 20:03:08.893715 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:08.893659    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": read tcp 10.132.0.2:50714->10.132.0.146:15021: read: connection reset by peer"
Nov 03 20:03:09.520734 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:09.520693    5051 generic.go:356] "Generic (PLEG): container finished" podID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerID="2cb51d86a176161a974bbae847b49fd46a3438e447501b811a5727dbc7994565" exitCode=0
Nov 03 20:03:09.520936 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:09.520771    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" event={"ID":"9089be58-dbe7-4c22-bb32-7006c5f86423","Type":"ContainerDied","Data":"2cb51d86a176161a974bbae847b49fd46a3438e447501b811a5727dbc7994565"}
Nov 03 20:03:09.521249 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:09.521231    5051 scope.go:122] "RemoveContainer" containerID="2cb51d86a176161a974bbae847b49fd46a3438e447501b811a5727dbc7994565"
Nov 03 20:03:09.522494 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:09.522466    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:03:09.522593 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:09.522529    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:03:09.522593 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:09.522560    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:03:10.528545 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:10.528508    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" event={"ID":"9089be58-dbe7-4c22-bb32-7006c5f86423","Type":"ContainerStarted","Data":"0efa35de04b6fead2256b829db02c772f2434259b9478873444cbdf4c02eb6f7"}
Nov 03 20:03:11.423682 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:11.423641    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:03:11.423928 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:11.423901    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Nov 03 20:03:11.424000 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:11.423961    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Nov 03 20:03:12.423433 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:12.423392    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Nov 03 20:03:12.423907 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:12.423450    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Nov 03 20:03:13.424290 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:13.424253    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Nov 03 20:03:13.424753 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:13.424308    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Nov 03 20:03:14.423716 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:14.423676    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Nov 03 20:03:14.423902 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:14.423738    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Nov 03 20:03:15.102378 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:15.102346    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:03:15.423941 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:15.423855    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Nov 03 20:03:15.423941 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:15.423917    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Nov 03 20:03:16.427415 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:16.427381    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:03:16.427818 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:16.427654    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:03:16.428250 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:16.428235    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:03:31.203009 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:31.202932    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:03:32.765783 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:03:32.765746    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:03:53.708362 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:03:53.708330    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:04:02.766693 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:04:02.766649    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:04:04.110365 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.110333    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"]
Nov 03 20:04:04.110869 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.110599    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5" podUID="3b156dad-d0ed-4ed1-89ea-682467e1ca39" containerName="maas-api" containerID="cri-o://78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d" gracePeriod=30
Nov 03 20:04:04.386649 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.386625    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:04:04.528304 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.528272    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b156dad-d0ed-4ed1-89ea-682467e1ca39-kube-api-access-vkhjh\" (UniqueName: \"kubernetes.io/projected/3b156dad-d0ed-4ed1-89ea-682467e1ca39-kube-api-access-vkhjh\") pod \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") "
Nov 03 20:04:04.528508 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.528315    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-metrics-certs\") pod \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") "
Nov 03 20:04:04.528508 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.528334    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b156dad-d0ed-4ed1-89ea-682467e1ca39-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b156dad-d0ed-4ed1-89ea-682467e1ca39-openshift-service-ca\") pod \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") "
Nov 03 20:04:04.528508 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.528377    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-maas-api-tls\") pod \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\" (UID: \"3b156dad-d0ed-4ed1-89ea-682467e1ca39\") "
Nov 03 20:04:04.528756 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.528727    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b156dad-d0ed-4ed1-89ea-682467e1ca39-openshift-service-ca" pod "3b156dad-d0ed-4ed1-89ea-682467e1ca39" (UID: "3b156dad-d0ed-4ed1-89ea-682467e1ca39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:04:04.530567 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.530540    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-metrics-certs" pod "3b156dad-d0ed-4ed1-89ea-682467e1ca39" (UID: "3b156dad-d0ed-4ed1-89ea-682467e1ca39"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:04:04.530742 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.530628    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-maas-api-tls" pod "3b156dad-d0ed-4ed1-89ea-682467e1ca39" (UID: "3b156dad-d0ed-4ed1-89ea-682467e1ca39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:04:04.530742 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.530654    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b156dad-d0ed-4ed1-89ea-682467e1ca39-kube-api-access-vkhjh" pod "3b156dad-d0ed-4ed1-89ea-682467e1ca39" (UID: "3b156dad-d0ed-4ed1-89ea-682467e1ca39"). InnerVolumeSpecName "kube-api-access-vkhjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:04:04.629494 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.629417    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkhjh\" (UniqueName: \"kubernetes.io/projected/3b156dad-d0ed-4ed1-89ea-682467e1ca39-kube-api-access-vkhjh\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:04.629494 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.629447    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:04.629494 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.629457    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b156dad-d0ed-4ed1-89ea-682467e1ca39-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:04.629494 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.629468    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b156dad-d0ed-4ed1-89ea-682467e1ca39-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:04.837924 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.837889    5051 generic.go:356] "Generic (PLEG): container finished" podID="3b156dad-d0ed-4ed1-89ea-682467e1ca39" containerID="78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d" exitCode=0
Nov 03 20:04:04.838126 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.837946    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"
Nov 03 20:04:04.838126 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.837973    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5" event={"ID":"3b156dad-d0ed-4ed1-89ea-682467e1ca39","Type":"ContainerDied","Data":"78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d"}
Nov 03 20:04:04.838126 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.838026    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5" event={"ID":"3b156dad-d0ed-4ed1-89ea-682467e1ca39","Type":"ContainerDied","Data":"9b2fcec124525fb588fb04f85b2dfd519659359cc595a5a3ccefbde44878a425"}
Nov 03 20:04:04.838126 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.838063    5051 scope.go:122] "RemoveContainer" containerID="78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d"
Nov 03 20:04:04.848475 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.848459    5051 scope.go:122] "RemoveContainer" containerID="78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d"
Nov 03 20:04:04.848719 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:04:04.848701    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d\": container with ID starting with 78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d not found: ID does not exist" containerID="78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d"
Nov 03 20:04:04.848779 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.848727    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d"} err="failed to get container status \"78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d\": rpc error: code = NotFound desc = could not find container \"78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d\": container with ID starting with 78195c7b0101037c3f3a56ef05680248bdbb66cf9e7f6018c881b13d4997141d not found: ID does not exist"
Nov 03 20:04:04.862085 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.862066    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"]
Nov 03 20:04:04.866052 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:04.866021    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-fcdf83-65457f9f74-6nhx5"]
Nov 03 20:04:05.227698 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:05.227661    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b156dad-d0ed-4ed1-89ea-682467e1ca39" path="/var/lib/kubelet/pods/3b156dad-d0ed-4ed1-89ea-682467e1ca39/volumes"
Nov 03 20:04:06.004992 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:05.998396    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:04:11.694893 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:11.694851    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"]
Nov 03 20:04:11.695540 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:11.695214    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy" containerID="cri-o://0efa35de04b6fead2256b829db02c772f2434259b9478873444cbdf4c02eb6f7" gracePeriod=30
Nov 03 20:04:16.430246 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:16.430209    5051 patch_prober.go:29] interesting pod/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 20:04:16.430661 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:16.430273    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 20:04:16.907577 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:16.907539    5051 generic.go:356] "Generic (PLEG): container finished" podID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerID="0efa35de04b6fead2256b829db02c772f2434259b9478873444cbdf4c02eb6f7" exitCode=0
Nov 03 20:04:16.907753 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:16.907609    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" event={"ID":"9089be58-dbe7-4c22-bb32-7006c5f86423","Type":"ContainerDied","Data":"0efa35de04b6fead2256b829db02c772f2434259b9478873444cbdf4c02eb6f7"}
Nov 03 20:04:16.907753 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:16.907704    5051 scope.go:122] "RemoveContainer" containerID="2cb51d86a176161a974bbae847b49fd46a3438e447501b811a5727dbc7994565"
Nov 03 20:04:16.966882 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:16.966826    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:04:17.050843 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.050813    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9089be58-dbe7-4c22-bb32-7006c5f86423-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9089be58-dbe7-4c22-bb32-7006c5f86423-istiod-ca-cert\") pod \"9089be58-dbe7-4c22-bb32-7006c5f86423\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") "
Nov 03 20:04:17.050993 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.050851    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-envoy\") pod \"9089be58-dbe7-4c22-bb32-7006c5f86423\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") "
Nov 03 20:04:17.050993 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.050871    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-token\" (UniqueName: \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-token\") pod \"9089be58-dbe7-4c22-bb32-7006c5f86423\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") "
Nov 03 20:04:17.050993 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.050908    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-podinfo\") pod \"9089be58-dbe7-4c22-bb32-7006c5f86423\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") "
Nov 03 20:04:17.050993 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.050933    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-kube-api-access-sfxm8\" (UniqueName: \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-kube-api-access-sfxm8\") pod \"9089be58-dbe7-4c22-bb32-7006c5f86423\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") "
Nov 03 20:04:17.050993 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.050991    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-certs\") pod \"9089be58-dbe7-4c22-bb32-7006c5f86423\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") "
Nov 03 20:04:17.051295 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.051049    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-data\") pod \"9089be58-dbe7-4c22-bb32-7006c5f86423\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") "
Nov 03 20:04:17.051295 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.051150    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-credential-socket\") pod \"9089be58-dbe7-4c22-bb32-7006c5f86423\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") "
Nov 03 20:04:17.051295 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.051196    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-socket\") pod \"9089be58-dbe7-4c22-bb32-7006c5f86423\" (UID: \"9089be58-dbe7-4c22-bb32-7006c5f86423\") "
Nov 03 20:04:17.051475 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.051369    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9089be58-dbe7-4c22-bb32-7006c5f86423-istiod-ca-cert" pod "9089be58-dbe7-4c22-bb32-7006c5f86423" (UID: "9089be58-dbe7-4c22-bb32-7006c5f86423"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:04:17.051591 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.051558    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-data" pod "9089be58-dbe7-4c22-bb32-7006c5f86423" (UID: "9089be58-dbe7-4c22-bb32-7006c5f86423"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:04:17.051661 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.051594    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9089be58-dbe7-4c22-bb32-7006c5f86423-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:17.051661 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.051613    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-certs" pod "9089be58-dbe7-4c22-bb32-7006c5f86423" (UID: "9089be58-dbe7-4c22-bb32-7006c5f86423"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:04:17.051797 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.051776    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-credential-socket" pod "9089be58-dbe7-4c22-bb32-7006c5f86423" (UID: "9089be58-dbe7-4c22-bb32-7006c5f86423"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:04:17.051865 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.051798    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-socket" pod "9089be58-dbe7-4c22-bb32-7006c5f86423" (UID: "9089be58-dbe7-4c22-bb32-7006c5f86423"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:04:17.053254 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.053224    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-podinfo" pod "9089be58-dbe7-4c22-bb32-7006c5f86423" (UID: "9089be58-dbe7-4c22-bb32-7006c5f86423"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 20:04:17.053351 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.053312    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-kube-api-access-sfxm8" pod "9089be58-dbe7-4c22-bb32-7006c5f86423" (UID: "9089be58-dbe7-4c22-bb32-7006c5f86423"). InnerVolumeSpecName "kube-api-access-sfxm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:04:17.053430 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.053415    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-envoy" pod "9089be58-dbe7-4c22-bb32-7006c5f86423" (UID: "9089be58-dbe7-4c22-bb32-7006c5f86423"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:04:17.053640 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.053626    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-token" pod "9089be58-dbe7-4c22-bb32-7006c5f86423" (UID: "9089be58-dbe7-4c22-bb32-7006c5f86423"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:04:17.112884 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.112843    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"]
Nov 03 20:04:17.113386 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.113367    5051 cpu_manager.go:414] "containerMap: removing container" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy"
Nov 03 20:04:17.113486 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.113388    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy"
Nov 03 20:04:17.113486 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.113402    5051 cpu_manager.go:414] "containerMap: removing container" podUID="3b156dad-d0ed-4ed1-89ea-682467e1ca39" containerName="maas-api"
Nov 03 20:04:17.113486 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.113409    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b156dad-d0ed-4ed1-89ea-682467e1ca39" containerName="maas-api"
Nov 03 20:04:17.113486 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.113453    5051 cpu_manager.go:414] "containerMap: removing container" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy"
Nov 03 20:04:17.113486 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.113461    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy"
Nov 03 20:04:17.113739 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.113576    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy"
Nov 03 20:04:17.113739 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.113588    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b156dad-d0ed-4ed1-89ea-682467e1ca39" containerName="maas-api"
Nov 03 20:04:17.113739 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.113607    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" containerName="istio-proxy"
Nov 03 20:04:17.117031 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.117010    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.121153 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.121131    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-9ea21f26-kserve-self-signed-certs\""
Nov 03 20:04:17.130125 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.130091    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"]
Nov 03 20:04:17.152796 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.152768    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5xqw\" (UniqueName: \"kubernetes.io/projected/338ec266-27ff-447d-8822-a8c9d1a5e099-kube-api-access-g5xqw\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.152933 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.152819    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-dshm\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.153010 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.152948    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/338ec266-27ff-447d-8822-a8c9d1a5e099-tls-certs\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.153010 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.152989    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-model-cache\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.153150 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.153027    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-tmp-dir\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.153150 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.153128    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-home\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.153263 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.153176    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:17.153263 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.153187    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:17.153263 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.153217    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:17.153263 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.153236    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:17.153263 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.153248    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:17.153263 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.153257    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfxm8\" (UniqueName: \"kubernetes.io/projected/9089be58-dbe7-4c22-bb32-7006c5f86423-kube-api-access-sfxm8\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:17.153573 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.153266    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:17.153573 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.153276    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9089be58-dbe7-4c22-bb32-7006c5f86423-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:04:17.254392 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.254357    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5xqw\" (UniqueName: \"kubernetes.io/projected/338ec266-27ff-447d-8822-a8c9d1a5e099-kube-api-access-g5xqw\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.254604 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.254436    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-dshm\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.254604 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.254479    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/338ec266-27ff-447d-8822-a8c9d1a5e099-tls-certs\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.254604 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.254511    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-model-cache\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.254604 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.254547    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-tmp-dir\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.254849 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.254647    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-home\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.255370 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.255187    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-model-cache\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.255370 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.255250    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-home\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.255370 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.255322    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-tmp-dir\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.257496 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.257476    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-dshm\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.257856 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.257837    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/338ec266-27ff-447d-8822-a8c9d1a5e099-tls-certs\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.267403 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.267380    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5xqw\" (UniqueName: \"kubernetes.io/projected/338ec266-27ff-447d-8822-a8c9d1a5e099-kube-api-access-g5xqw\") pod \"e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") " pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.432591 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.432561    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:17.706630 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.706603    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"]
Nov 03 20:04:17.708009 ip-10-0-143-212 kubenswrapper[5051]: W1103 20:04:17.707980    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod338ec266_27ff_447d_8822_a8c9d1a5e099.slice/crio-d93116fbd1a9ad08cb858eb68b29e57f3f12116fdc7973797f58cd6f0ebcd562 WatchSource:0}: Error finding container d93116fbd1a9ad08cb858eb68b29e57f3f12116fdc7973797f58cd6f0ebcd562: Status 404 returned error can't find the container with id d93116fbd1a9ad08cb858eb68b29e57f3f12116fdc7973797f58cd6f0ebcd562
Nov 03 20:04:17.913299 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.913270    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"
Nov 03 20:04:17.913492 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.913303    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w" event={"ID":"9089be58-dbe7-4c22-bb32-7006c5f86423","Type":"ContainerDied","Data":"73e0a28c6f0e798a0516929558cba98e6363d80b5562b1750c7eabc20ee8c610"}
Nov 03 20:04:17.913492 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.913371    5051 scope.go:122] "RemoveContainer" containerID="0efa35de04b6fead2256b829db02c772f2434259b9478873444cbdf4c02eb6f7"
Nov 03 20:04:17.915031 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.915003    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp" event={"ID":"338ec266-27ff-447d-8822-a8c9d1a5e099","Type":"ContainerStarted","Data":"240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001"}
Nov 03 20:04:17.915225 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.915063    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp" event={"ID":"338ec266-27ff-447d-8822-a8c9d1a5e099","Type":"ContainerStarted","Data":"d93116fbd1a9ad08cb858eb68b29e57f3f12116fdc7973797f58cd6f0ebcd562"}
Nov 03 20:04:17.943221 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.939762    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp" podStartSLOduration=0.939734882 podStartE2EDuration="939.734882ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:04:17.936713345 +0000 UTC m=+2258.844735283" watchObservedRunningTime="2026-11-03 20:04:17.939734882 +0000 UTC m=+2258.847756821"
Nov 03 20:04:17.954290 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.954267    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"]
Nov 03 20:04:17.957794 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:17.957735    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-fcdf83-openshift-default-587b7c7fccdct6w"]
Nov 03 20:04:18.433107 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:18.433073    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:18.907330 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:18.907301    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:04:19.227891 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:19.227859    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9089be58-dbe7-4c22-bb32-7006c5f86423" path="/var/lib/kubelet/pods/9089be58-dbe7-4c22-bb32-7006c5f86423/volumes"
Nov 03 20:04:27.432980 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:27.432948    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:27.445517 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:27.445484    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:27.979444 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:27.979415    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:04:32.315841 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.315805    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"]
Nov 03 20:04:32.321189 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.321166    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.329493 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.329463    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"]
Nov 03 20:04:32.335557 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.335536    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-9ea21f26-kserve-self-signed-certs\""
Nov 03 20:04:32.450110 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.450022    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-model-cache\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.450475 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.450449    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tmp-dir\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.450781 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.450750    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tls-certs\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.450900 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.450823    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85gpr\" (UniqueName: \"kubernetes.io/projected/5b23744f-c517-406f-a1ab-c478a4e3bdc3-kube-api-access-85gpr\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.450900 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.450864    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-dshm\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.451011 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.450919    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-home\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.552255 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.552220    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-model-cache\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.552255 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.552261    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tmp-dir\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.552494 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.552311    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tls-certs\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.552494 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.552341    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85gpr\" (UniqueName: \"kubernetes.io/projected/5b23744f-c517-406f-a1ab-c478a4e3bdc3-kube-api-access-85gpr\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.552494 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.552365    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-dshm\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.552494 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.552395    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-home\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.552734 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.552704    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-model-cache\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.552734 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.552728    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tmp-dir\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.552849 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.552752    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-home\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.554762 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.554739    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-dshm\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.554904 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.554887    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tls-certs\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.564553 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.564528    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85gpr\" (UniqueName: \"kubernetes.io/projected/5b23744f-c517-406f-a1ab-c478a4e3bdc3-kube-api-access-85gpr\") pod \"e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") " pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.634739 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.634659    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:32.768763 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:04:32.768725    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:04:32.910054 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.910016    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"]
Nov 03 20:04:32.912198 ip-10-0-143-212 kubenswrapper[5051]: W1103 20:04:32.912165    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b23744f_c517_406f_a1ab_c478a4e3bdc3.slice/crio-595858090b4263daf45b05cfca7da33cf3aeea0fdfc7a6d8761dda95bc35ee72 WatchSource:0}: Error finding container 595858090b4263daf45b05cfca7da33cf3aeea0fdfc7a6d8761dda95bc35ee72: Status 404 returned error can't find the container with id 595858090b4263daf45b05cfca7da33cf3aeea0fdfc7a6d8761dda95bc35ee72
Nov 03 20:04:32.996194 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.996163    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn" event={"ID":"5b23744f-c517-406f-a1ab-c478a4e3bdc3","Type":"ContainerStarted","Data":"5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af"}
Nov 03 20:04:32.996354 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:32.996212    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn" event={"ID":"5b23744f-c517-406f-a1ab-c478a4e3bdc3","Type":"ContainerStarted","Data":"595858090b4263daf45b05cfca7da33cf3aeea0fdfc7a6d8761dda95bc35ee72"}
Nov 03 20:04:33.017009 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:33.016964    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn" podStartSLOduration=1.016950352 podStartE2EDuration="1.016950352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:04:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:04:33.015722308 +0000 UTC m=+2273.923744265" watchObservedRunningTime="2026-11-03 20:04:33.016950352 +0000 UTC m=+2273.924972290"
Nov 03 20:04:33.635166 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:33.635133    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:34.284299 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:34.284261    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:04:42.635842 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:42.635807    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:42.648474 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:42.648448    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:43.069156 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:43.069122    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:04:50.256582 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:04:50.256549    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:05:00.454723 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:00.454684    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"]
Nov 03 20:05:01.770107 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:01.770070    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"]
Nov 03 20:05:01.972754 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:01.972719    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:05:02.773310 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:05:02.773266    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:05:07.820332 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:07.820296    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:05:14.302555 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:14.302518    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:05:15.481571 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:15.481483    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp" podUID="338ec266-27ff-447d-8822-a8c9d1a5e099" containerName="main" containerID="cri-o://240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001" gracePeriod=45
Nov 03 20:05:16.034133 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.034071    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:05:16.130244 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.130211    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/338ec266-27ff-447d-8822-a8c9d1a5e099-kube-api-access-g5xqw\" (UniqueName: \"kubernetes.io/projected/338ec266-27ff-447d-8822-a8c9d1a5e099-kube-api-access-g5xqw\") pod \"338ec266-27ff-447d-8822-a8c9d1a5e099\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") "
Nov 03 20:05:16.130447 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.130308    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-tmp-dir\") pod \"338ec266-27ff-447d-8822-a8c9d1a5e099\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") "
Nov 03 20:05:16.130447 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.130370    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-home\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-home\") pod \"338ec266-27ff-447d-8822-a8c9d1a5e099\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") "
Nov 03 20:05:16.130447 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.130417    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-dshm\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-dshm\") pod \"338ec266-27ff-447d-8822-a8c9d1a5e099\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") "
Nov 03 20:05:16.130447 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.130445    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-model-cache\") pod \"338ec266-27ff-447d-8822-a8c9d1a5e099\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") "
Nov 03 20:05:16.130708 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.130490    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/338ec266-27ff-447d-8822-a8c9d1a5e099-tls-certs\" (UniqueName: \"kubernetes.io/secret/338ec266-27ff-447d-8822-a8c9d1a5e099-tls-certs\") pod \"338ec266-27ff-447d-8822-a8c9d1a5e099\" (UID: \"338ec266-27ff-447d-8822-a8c9d1a5e099\") "
Nov 03 20:05:16.130708 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.130597    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-tmp-dir" pod "338ec266-27ff-447d-8822-a8c9d1a5e099" (UID: "338ec266-27ff-447d-8822-a8c9d1a5e099"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:05:16.130873 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.130854    5051 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-tmp-dir\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:16.131104 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.131084    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-model-cache" pod "338ec266-27ff-447d-8822-a8c9d1a5e099" (UID: "338ec266-27ff-447d-8822-a8c9d1a5e099"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:05:16.131193 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.131169    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-home" pod "338ec266-27ff-447d-8822-a8c9d1a5e099" (UID: "338ec266-27ff-447d-8822-a8c9d1a5e099"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:05:16.133089 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.133068    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/338ec266-27ff-447d-8822-a8c9d1a5e099-tls-certs" pod "338ec266-27ff-447d-8822-a8c9d1a5e099" (UID: "338ec266-27ff-447d-8822-a8c9d1a5e099"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:05:16.134822 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.134796    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/338ec266-27ff-447d-8822-a8c9d1a5e099-kube-api-access-g5xqw" pod "338ec266-27ff-447d-8822-a8c9d1a5e099" (UID: "338ec266-27ff-447d-8822-a8c9d1a5e099"). InnerVolumeSpecName "kube-api-access-g5xqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:05:16.135908 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.135885    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-dshm" pod "338ec266-27ff-447d-8822-a8c9d1a5e099" (UID: "338ec266-27ff-447d-8822-a8c9d1a5e099"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:05:16.231506 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.231469    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5xqw\" (UniqueName: \"kubernetes.io/projected/338ec266-27ff-447d-8822-a8c9d1a5e099-kube-api-access-g5xqw\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:16.231506 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.231496    5051 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-home\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:16.231506 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.231505    5051 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-dshm\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:16.231506 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.231513    5051 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/338ec266-27ff-447d-8822-a8c9d1a5e099-model-cache\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:16.231834 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.231523    5051 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/338ec266-27ff-447d-8822-a8c9d1a5e099-tls-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:16.312859 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.312673    5051 generic.go:356] "Generic (PLEG): container finished" podID="338ec266-27ff-447d-8822-a8c9d1a5e099" containerID="240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001" exitCode=0
Nov 03 20:05:16.313148 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.312746    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"
Nov 03 20:05:16.313404 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.312768    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp" event={"ID":"338ec266-27ff-447d-8822-a8c9d1a5e099","Type":"ContainerDied","Data":"240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001"}
Nov 03 20:05:16.313404 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.313270    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp" event={"ID":"338ec266-27ff-447d-8822-a8c9d1a5e099","Type":"ContainerDied","Data":"d93116fbd1a9ad08cb858eb68b29e57f3f12116fdc7973797f58cd6f0ebcd562"}
Nov 03 20:05:16.313404 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.313294    5051 scope.go:122] "RemoveContainer" containerID="240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001"
Nov 03 20:05:16.325818 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.325799    5051 scope.go:122] "RemoveContainer" containerID="240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001"
Nov 03 20:05:16.326211 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:05:16.326189    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001\": container with ID starting with 240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001 not found: ID does not exist" containerID="240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001"
Nov 03 20:05:16.326284 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.326223    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001"} err="failed to get container status \"240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001\": rpc error: code = NotFound desc = could not find container \"240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001\": container with ID starting with 240a283e64ea5f799330fdfea53d4284fde17b93bf5441c5486b0eced3be4001 not found: ID does not exist"
Nov 03 20:05:16.338874 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.338851    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"]
Nov 03 20:05:16.343424 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.343399    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp"]
Nov 03 20:05:16.791018 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.790975    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn" podUID="5b23744f-c517-406f-a1ab-c478a4e3bdc3" containerName="main" containerID="cri-o://5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af" gracePeriod=45
Nov 03 20:05:16.976253 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:16.976214    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-a-9ea21f26-kserve-8f7844857-x5cfp" podUID="338ec266-27ff-447d-8822-a8c9d1a5e099" containerName="main" probeResult="failure" output="Get \"https://10.132.0.148:8000/ready\": context deadline exceeded"
Nov 03 20:05:17.052141 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.052089    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:05:17.140816 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.140785    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b23744f-c517-406f-a1ab-c478a4e3bdc3-kube-api-access-85gpr\" (UniqueName: \"kubernetes.io/projected/5b23744f-c517-406f-a1ab-c478a4e3bdc3-kube-api-access-85gpr\") pod \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") "
Nov 03 20:05:17.140978 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.140834    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tls-certs\") pod \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") "
Nov 03 20:05:17.140978 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.140862    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-model-cache\") pod \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") "
Nov 03 20:05:17.140978 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.140929    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-dshm\") pod \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") "
Nov 03 20:05:17.141195 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.140989    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tmp-dir\") pod \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") "
Nov 03 20:05:17.141195 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.141098    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-home\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-home\") pod \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\" (UID: \"5b23744f-c517-406f-a1ab-c478a4e3bdc3\") "
Nov 03 20:05:17.141195 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.141171    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-model-cache" pod "5b23744f-c517-406f-a1ab-c478a4e3bdc3" (UID: "5b23744f-c517-406f-a1ab-c478a4e3bdc3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:05:17.141334 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.141268    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tmp-dir" pod "5b23744f-c517-406f-a1ab-c478a4e3bdc3" (UID: "5b23744f-c517-406f-a1ab-c478a4e3bdc3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:05:17.141403 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.141370    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-home" pod "5b23744f-c517-406f-a1ab-c478a4e3bdc3" (UID: "5b23744f-c517-406f-a1ab-c478a4e3bdc3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:05:17.141560 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.141543    5051 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tmp-dir\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:17.141622 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.141563    5051 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-home\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:17.141622 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.141572    5051 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-model-cache\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:17.142987 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.142966    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b23744f-c517-406f-a1ab-c478a4e3bdc3-kube-api-access-85gpr" pod "5b23744f-c517-406f-a1ab-c478a4e3bdc3" (UID: "5b23744f-c517-406f-a1ab-c478a4e3bdc3"). InnerVolumeSpecName "kube-api-access-85gpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:05:17.143465 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.143447    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-dshm" pod "5b23744f-c517-406f-a1ab-c478a4e3bdc3" (UID: "5b23744f-c517-406f-a1ab-c478a4e3bdc3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:05:17.143465 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.143455    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tls-certs" pod "5b23744f-c517-406f-a1ab-c478a4e3bdc3" (UID: "5b23744f-c517-406f-a1ab-c478a4e3bdc3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:05:17.228232 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.228203    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="338ec266-27ff-447d-8822-a8c9d1a5e099" path="/var/lib/kubelet/pods/338ec266-27ff-447d-8822-a8c9d1a5e099/volumes"
Nov 03 20:05:17.242074 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.242051    5051 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b23744f-c517-406f-a1ab-c478a4e3bdc3-dshm\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:17.242074 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.242076    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85gpr\" (UniqueName: \"kubernetes.io/projected/5b23744f-c517-406f-a1ab-c478a4e3bdc3-kube-api-access-85gpr\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:17.242227 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.242086    5051 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b23744f-c517-406f-a1ab-c478a4e3bdc3-tls-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:17.324488 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.324415    5051 generic.go:356] "Generic (PLEG): container finished" podID="5b23744f-c517-406f-a1ab-c478a4e3bdc3" containerID="5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af" exitCode=0
Nov 03 20:05:17.324488 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.324455    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn" event={"ID":"5b23744f-c517-406f-a1ab-c478a4e3bdc3","Type":"ContainerDied","Data":"5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af"}
Nov 03 20:05:17.324488 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.324459    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"
Nov 03 20:05:17.324488 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.324479    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn" event={"ID":"5b23744f-c517-406f-a1ab-c478a4e3bdc3","Type":"ContainerDied","Data":"595858090b4263daf45b05cfca7da33cf3aeea0fdfc7a6d8761dda95bc35ee72"}
Nov 03 20:05:17.324729 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.324496    5051 scope.go:122] "RemoveContainer" containerID="5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af"
Nov 03 20:05:17.340144 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.340121    5051 scope.go:122] "RemoveContainer" containerID="5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af"
Nov 03 20:05:17.340403 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:05:17.340382    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af\": container with ID starting with 5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af not found: ID does not exist" containerID="5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af"
Nov 03 20:05:17.340456 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.340410    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af"} err="failed to get container status \"5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af\": rpc error: code = NotFound desc = could not find container \"5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af\": container with ID starting with 5af249b5eb3b7f8a0a58fd4b4fe79faf7b57c537faff20f610d972a4213126af not found: ID does not exist"
Nov 03 20:05:17.348815 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.348794    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"]
Nov 03 20:05:17.352090 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:17.352068    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-9ea21f26-kserve-7c9fd8cdb9-4rvbn"]
Nov 03 20:05:19.228100 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:19.228037    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b23744f-c517-406f-a1ab-c478a4e3bdc3" path="/var/lib/kubelet/pods/5b23744f-c517-406f-a1ab-c478a4e3bdc3/volumes"
Nov 03 20:05:19.478064 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:19.464597    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:05:32.766020 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:05:32.765981    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:05:34.863001 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:34.862964    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:05:35.968927 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:35.968889    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"]
Nov 03 20:05:35.969625 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:35.969598    5051 cpu_manager.go:414] "containerMap: removing container" podUID="5b23744f-c517-406f-a1ab-c478a4e3bdc3" containerName="main"
Nov 03 20:05:35.969625 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:35.969622    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b23744f-c517-406f-a1ab-c478a4e3bdc3" containerName="main"
Nov 03 20:05:35.969746 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:35.969649    5051 cpu_manager.go:414] "containerMap: removing container" podUID="338ec266-27ff-447d-8822-a8c9d1a5e099" containerName="main"
Nov 03 20:05:35.969746 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:35.969657    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="338ec266-27ff-447d-8822-a8c9d1a5e099" containerName="main"
Nov 03 20:05:35.969808 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:35.969775    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b23744f-c517-406f-a1ab-c478a4e3bdc3" containerName="main"
Nov 03 20:05:35.969808 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:35.969790    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="338ec266-27ff-447d-8822-a8c9d1a5e099" containerName="main"
Nov 03 20:05:35.973490 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:35.973469    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:35.986748 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:35.986720    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"]
Nov 03 20:05:36.063754 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.063722    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb932f93-3367-4339-95d0-6408d13a3cf8-maas-api-tls\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.063926 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.063807    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb932f93-3367-4339-95d0-6408d13a3cf8-metrics-certs\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.063926 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.063876    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb932f93-3367-4339-95d0-6408d13a3cf8-openshift-service-ca\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.064012 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.063981    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clnk9\" (UniqueName: \"kubernetes.io/projected/bb932f93-3367-4339-95d0-6408d13a3cf8-kube-api-access-clnk9\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.165575 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.165540    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb932f93-3367-4339-95d0-6408d13a3cf8-metrics-certs\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.165746 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.165602    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb932f93-3367-4339-95d0-6408d13a3cf8-openshift-service-ca\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.165746 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.165645    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clnk9\" (UniqueName: \"kubernetes.io/projected/bb932f93-3367-4339-95d0-6408d13a3cf8-kube-api-access-clnk9\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.165746 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.165719    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb932f93-3367-4339-95d0-6408d13a3cf8-maas-api-tls\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.166371 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.166345    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb932f93-3367-4339-95d0-6408d13a3cf8-openshift-service-ca\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.168262 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.168230    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb932f93-3367-4339-95d0-6408d13a3cf8-metrics-certs\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.168262 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.168257    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb932f93-3367-4339-95d0-6408d13a3cf8-maas-api-tls\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.178411 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.178387    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clnk9\" (UniqueName: \"kubernetes.io/projected/bb932f93-3367-4339-95d0-6408d13a3cf8-kube-api-access-clnk9\") pod \"maas-api-755bc5f869-stjv7\" (UID: \"bb932f93-3367-4339-95d0-6408d13a3cf8\") " pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.288941 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.288856    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:36.568301 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.568279    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"]
Nov 03 20:05:36.570472 ip-10-0-143-212 kubenswrapper[5051]: W1103 20:05:36.570443    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb932f93_3367_4339_95d0_6408d13a3cf8.slice/crio-ee2e19328e90d7cda3fdc5e8f18e274c52fb9a6286d850b8ce70fee1fe2a372b WatchSource:0}: Error finding container ee2e19328e90d7cda3fdc5e8f18e274c52fb9a6286d850b8ce70fee1fe2a372b: Status 404 returned error can't find the container with id ee2e19328e90d7cda3fdc5e8f18e274c52fb9a6286d850b8ce70fee1fe2a372b
Nov 03 20:05:36.571735 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:36.571717    5051 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 20:05:37.420281 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:37.420198    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7" event={"ID":"bb932f93-3367-4339-95d0-6408d13a3cf8","Type":"ContainerStarted","Data":"dd13b183cf3ce2dfe28063ea2d5050ee142defc2c2f1229e1bc858d24db02c2b"}
Nov 03 20:05:37.420281 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:37.420254    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7" event={"ID":"bb932f93-3367-4339-95d0-6408d13a3cf8","Type":"ContainerStarted","Data":"ee2e19328e90d7cda3fdc5e8f18e274c52fb9a6286d850b8ce70fee1fe2a372b"}
Nov 03 20:05:37.420686 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:37.420307    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:37.442380 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:37.442335    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7" podStartSLOduration=1.902982161 podStartE2EDuration="2.442321566s" totalImagesPullingTime="539.339405ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:05:37.440624634 +0000 UTC m=+2338.348646587" watchObservedRunningTime="2026-11-03 20:05:37.442321566 +0000 UTC m=+2338.350343497"
Nov 03 20:05:43.431554 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:43.431527    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-755bc5f869-stjv7"
Nov 03 20:05:43.517877 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:43.517838    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"]
Nov 03 20:05:43.518152 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:43.518104    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz" podUID="10ecdc8b-d2bc-4019-86d8-9a0847d72cfb" containerName="maas-api" containerID="cri-o://c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc" gracePeriod=30
Nov 03 20:05:44.777328 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:44.777304    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 20:05:44.953316 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:44.953232    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-kube-api-access-jgc9r\" (UniqueName: \"kubernetes.io/projected/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-kube-api-access-jgc9r\") pod \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") "
Nov 03 20:05:44.953316 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:44.953307    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-maas-api-tls\") pod \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") "
Nov 03 20:05:44.953539 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:44.953337    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-openshift-service-ca\") pod \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") "
Nov 03 20:05:44.953539 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:44.953362    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-metrics-certs\") pod \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\" (UID: \"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb\") "
Nov 03 20:05:44.953722 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:44.953694    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-openshift-service-ca" pod "10ecdc8b-d2bc-4019-86d8-9a0847d72cfb" (UID: "10ecdc8b-d2bc-4019-86d8-9a0847d72cfb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:05:44.955440 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:44.955418    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-kube-api-access-jgc9r" pod "10ecdc8b-d2bc-4019-86d8-9a0847d72cfb" (UID: "10ecdc8b-d2bc-4019-86d8-9a0847d72cfb"). InnerVolumeSpecName "kube-api-access-jgc9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:05:44.955527 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:44.955471    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-metrics-certs" pod "10ecdc8b-d2bc-4019-86d8-9a0847d72cfb" (UID: "10ecdc8b-d2bc-4019-86d8-9a0847d72cfb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:05:44.955641 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:44.955622    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-maas-api-tls" pod "10ecdc8b-d2bc-4019-86d8-9a0847d72cfb" (UID: "10ecdc8b-d2bc-4019-86d8-9a0847d72cfb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:05:45.054279 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.054252    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgc9r\" (UniqueName: \"kubernetes.io/projected/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-kube-api-access-jgc9r\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:45.054279 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.054279    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:45.054502 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.054293    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:45.054502 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.054306    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:05:45.462352 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.462313    5051 generic.go:356] "Generic (PLEG): container finished" podID="10ecdc8b-d2bc-4019-86d8-9a0847d72cfb" containerID="c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc" exitCode=0
Nov 03 20:05:45.462658 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.462365    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"
Nov 03 20:05:45.462658 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.462403    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz" event={"ID":"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb","Type":"ContainerDied","Data":"c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc"}
Nov 03 20:05:45.462658 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.462466    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz" event={"ID":"10ecdc8b-d2bc-4019-86d8-9a0847d72cfb","Type":"ContainerDied","Data":"1f6e506b9aeee49a2d987248a0768f8ab579911fc43b569005567a5df279a85a"}
Nov 03 20:05:45.462658 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.462496    5051 scope.go:122] "RemoveContainer" containerID="c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc"
Nov 03 20:05:45.474008 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.473976    5051 scope.go:122] "RemoveContainer" containerID="c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc"
Nov 03 20:05:45.474304 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:05:45.474283    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc\": container with ID starting with c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc not found: ID does not exist" containerID="c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc"
Nov 03 20:05:45.474381 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.474318    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc"} err="failed to get container status \"c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc\": rpc error: code = NotFound desc = could not find container \"c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc\": container with ID starting with c0eb4e06ad092000b853b1382a3316439646162df3eb438218be58d45cc153cc not found: ID does not exist"
Nov 03 20:05:45.481232 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.481209    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"]
Nov 03 20:05:45.485063 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:45.485029    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-57c44bc9cc-rzrqz"]
Nov 03 20:05:47.227529 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.227491    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10ecdc8b-d2bc-4019-86d8-9a0847d72cfb" path="/var/lib/kubelet/pods/10ecdc8b-d2bc-4019-86d8-9a0847d72cfb/volumes"
Nov 03 20:05:47.256394 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.256361    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"]
Nov 03 20:05:47.257110 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.257089    5051 cpu_manager.go:414] "containerMap: removing container" podUID="10ecdc8b-d2bc-4019-86d8-9a0847d72cfb" containerName="maas-api"
Nov 03 20:05:47.257162 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.257115    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10ecdc8b-d2bc-4019-86d8-9a0847d72cfb" containerName="maas-api"
Nov 03 20:05:47.257268 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.257256    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="10ecdc8b-d2bc-4019-86d8-9a0847d72cfb" containerName="maas-api"
Nov 03 20:05:47.262358 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.262336    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.272773 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.272741    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9eef14db-serving-cert\""
Nov 03 20:05:47.278415 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.278389    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"]
Nov 03 20:05:47.379151 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.379111    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz52l\" (UniqueName: \"kubernetes.io/projected/972078b9-e4ff-42d0-b392-1ba5593823f2-kube-api-access-bz52l\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.379350 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.379187    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-metrics-certs\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.379478 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.379464    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/972078b9-e4ff-42d0-b392-1ba5593823f2-openshift-service-ca\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.379551 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.379492    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-maas-api-tls\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.480856 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.480742    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/972078b9-e4ff-42d0-b392-1ba5593823f2-openshift-service-ca\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.480856 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.480798    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-maas-api-tls\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.480856 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.480853    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz52l\" (UniqueName: \"kubernetes.io/projected/972078b9-e4ff-42d0-b392-1ba5593823f2-kube-api-access-bz52l\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.481136 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.480896    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-metrics-certs\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.481791 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.481758    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/972078b9-e4ff-42d0-b392-1ba5593823f2-openshift-service-ca\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.484449 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.484402    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-maas-api-tls\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.488829 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.488803    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-metrics-certs\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.493999 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.493967    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz52l\" (UniqueName: \"kubernetes.io/projected/972078b9-e4ff-42d0-b392-1ba5593823f2-kube-api-access-bz52l\") pod \"maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.580581 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.580546    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:47.855770 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:47.855746    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"]
Nov 03 20:05:47.857995 ip-10-0-143-212 kubenswrapper[5051]: W1103 20:05:47.857970    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod972078b9_e4ff_42d0_b392_1ba5593823f2.slice/crio-8c8bd4d665864fff727e23c810819474f21d48acadc6d0caa39bc9808c6d03e8 WatchSource:0}: Error finding container 8c8bd4d665864fff727e23c810819474f21d48acadc6d0caa39bc9808c6d03e8: Status 404 returned error can't find the container with id 8c8bd4d665864fff727e23c810819474f21d48acadc6d0caa39bc9808c6d03e8
Nov 03 20:05:48.481094 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:48.481054    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp" event={"ID":"972078b9-e4ff-42d0-b392-1ba5593823f2","Type":"ContainerStarted","Data":"39ca437c09b56dd00eeff889c89137abd0a5bcc7731e32f9eceb6c69b04fdf31"}
Nov 03 20:05:48.481489 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:48.481146    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:05:48.481489 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:48.481164    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp" event={"ID":"972078b9-e4ff-42d0-b392-1ba5593823f2","Type":"ContainerStarted","Data":"8c8bd4d665864fff727e23c810819474f21d48acadc6d0caa39bc9808c6d03e8"}
Nov 03 20:05:48.506309 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:48.506229    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp" podStartSLOduration=1.115176197 podStartE2EDuration="1.506215013s" totalImagesPullingTime="391.038816ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:05:48.503170464 +0000 UTC m=+2349.411192440" watchObservedRunningTime="2026-11-03 20:05:48.506215013 +0000 UTC m=+2349.414236952"
Nov 03 20:05:49.969055 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:49.969011    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:05:54.492424 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:05:54.492393    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:06:02.767318 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:06:02.767279    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:06:07.452826 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.452783    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"]
Nov 03 20:06:07.453377 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.452985    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp" podUID="972078b9-e4ff-42d0-b392-1ba5593823f2" containerName="maas-api" containerID="cri-o://39ca437c09b56dd00eeff889c89137abd0a5bcc7731e32f9eceb6c69b04fdf31" gracePeriod=30
Nov 03 20:06:07.576289 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.576260    5051 generic.go:356] "Generic (PLEG): container finished" podID="972078b9-e4ff-42d0-b392-1ba5593823f2" containerID="39ca437c09b56dd00eeff889c89137abd0a5bcc7731e32f9eceb6c69b04fdf31" exitCode=0
Nov 03 20:06:07.576433 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.576334    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp" event={"ID":"972078b9-e4ff-42d0-b392-1ba5593823f2","Type":"ContainerDied","Data":"39ca437c09b56dd00eeff889c89137abd0a5bcc7731e32f9eceb6c69b04fdf31"}
Nov 03 20:06:07.712356 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.712337    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:06:07.779967 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.779934    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-metrics-certs\") pod \"972078b9-e4ff-42d0-b392-1ba5593823f2\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") "
Nov 03 20:06:07.780172 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.780065    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/972078b9-e4ff-42d0-b392-1ba5593823f2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/972078b9-e4ff-42d0-b392-1ba5593823f2-openshift-service-ca\") pod \"972078b9-e4ff-42d0-b392-1ba5593823f2\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") "
Nov 03 20:06:07.780172 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.780090    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-maas-api-tls\") pod \"972078b9-e4ff-42d0-b392-1ba5593823f2\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") "
Nov 03 20:06:07.780172 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.780116    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/972078b9-e4ff-42d0-b392-1ba5593823f2-kube-api-access-bz52l\" (UniqueName: \"kubernetes.io/projected/972078b9-e4ff-42d0-b392-1ba5593823f2-kube-api-access-bz52l\") pod \"972078b9-e4ff-42d0-b392-1ba5593823f2\" (UID: \"972078b9-e4ff-42d0-b392-1ba5593823f2\") "
Nov 03 20:06:07.780505 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.780473    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/972078b9-e4ff-42d0-b392-1ba5593823f2-openshift-service-ca" pod "972078b9-e4ff-42d0-b392-1ba5593823f2" (UID: "972078b9-e4ff-42d0-b392-1ba5593823f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:06:07.782221 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.782194    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-maas-api-tls" pod "972078b9-e4ff-42d0-b392-1ba5593823f2" (UID: "972078b9-e4ff-42d0-b392-1ba5593823f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:06:07.782221 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.782212    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-metrics-certs" pod "972078b9-e4ff-42d0-b392-1ba5593823f2" (UID: "972078b9-e4ff-42d0-b392-1ba5593823f2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:06:07.782350 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.782194    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/972078b9-e4ff-42d0-b392-1ba5593823f2-kube-api-access-bz52l" pod "972078b9-e4ff-42d0-b392-1ba5593823f2" (UID: "972078b9-e4ff-42d0-b392-1ba5593823f2"). InnerVolumeSpecName "kube-api-access-bz52l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:06:07.881620 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.881586    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:06:07.881620 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.881616    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/972078b9-e4ff-42d0-b392-1ba5593823f2-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:06:07.881620 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.881626    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/972078b9-e4ff-42d0-b392-1ba5593823f2-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:06:07.881858 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:07.881636    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz52l\" (UniqueName: \"kubernetes.io/projected/972078b9-e4ff-42d0-b392-1ba5593823f2-kube-api-access-bz52l\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:06:08.583147 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:08.583104    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp" event={"ID":"972078b9-e4ff-42d0-b392-1ba5593823f2","Type":"ContainerDied","Data":"8c8bd4d665864fff727e23c810819474f21d48acadc6d0caa39bc9808c6d03e8"}
Nov 03 20:06:08.583603 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:08.583167    5051 scope.go:122] "RemoveContainer" containerID="39ca437c09b56dd00eeff889c89137abd0a5bcc7731e32f9eceb6c69b04fdf31"
Nov 03 20:06:08.583603 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:08.583124    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"
Nov 03 20:06:08.610321 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:08.610295    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"]
Nov 03 20:06:08.614079 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:08.614052    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eef14db-5f868fd6dc-x8kbp"]
Nov 03 20:06:09.093251 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:09.093218    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:06:09.227917 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:09.227884    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="972078b9-e4ff-42d0-b392-1ba5593823f2" path="/var/lib/kubelet/pods/972078b9-e4ff-42d0-b392-1ba5593823f2/volumes"
Nov 03 20:06:14.945182 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:14.945140    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"]
Nov 03 20:06:14.945636 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:14.945599    5051 cpu_manager.go:414] "containerMap: removing container" podUID="972078b9-e4ff-42d0-b392-1ba5593823f2" containerName="maas-api"
Nov 03 20:06:14.945636 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:14.945611    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="972078b9-e4ff-42d0-b392-1ba5593823f2" containerName="maas-api"
Nov 03 20:06:14.945726 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:14.945715    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="972078b9-e4ff-42d0-b392-1ba5593823f2" containerName="maas-api"
Nov 03 20:06:14.950364 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:14.950339    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:14.954551 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:14.954518    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-66e889-openshift-default-dockercfg-hhcsq\""
Nov 03 20:06:14.966589 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:14.966565    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"]
Nov 03 20:06:15.055245 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.055209    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-credential-socket\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.055245 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.055257    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-socket\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.055508 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.055356    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-envoy\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.055508 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.055408    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-podinfo\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.055613 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.055511    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-data\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.055613 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.055567    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdscc\" (UniqueName: \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-kube-api-access-zdscc\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.055717 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.055650    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-token\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.055717 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.055692    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-certs\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.055829 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.055774    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istiod-ca-cert\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.156780 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.156747    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdscc\" (UniqueName: \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-kube-api-access-zdscc\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.156951 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.156790    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-token\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.156951 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.156812    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-certs\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.156951 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.156867    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istiod-ca-cert\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.156951 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.156933    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-credential-socket\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.157224 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.156958    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-socket\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.157224 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.156982    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-envoy\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.157224 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.157011    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-podinfo\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.157224 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.157052    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-data\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.157453 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.157370    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-certs\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.157453 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.157413    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-socket\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.157577 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.157462    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-credential-socket\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.157577 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.157500    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-data\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.157818 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.157794    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istiod-ca-cert\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.159442 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.159420    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-envoy\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.159614 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.159592    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-podinfo\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.185012 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.184981    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-token\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.185242 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.185222    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdscc\" (UniqueName: \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-kube-api-access-zdscc\") pod \"e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") " pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.266692 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.266652    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:15.545275 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.545249    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"]
Nov 03 20:06:15.546677 ip-10-0-143-212 kubenswrapper[5051]: W1103 20:06:15.546649    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ae35a9_7c97_4620_bf14_ef380e2090fd.slice/crio-86d43d55f69fef80c4c22a9f78e6877e42943a3e6195c403bd6e1ddc8d77bbef WatchSource:0}: Error finding container 86d43d55f69fef80c4c22a9f78e6877e42943a3e6195c403bd6e1ddc8d77bbef: Status 404 returned error can't find the container with id 86d43d55f69fef80c4c22a9f78e6877e42943a3e6195c403bd6e1ddc8d77bbef
Nov 03 20:06:15.548842 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.548812    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:06:15.548921 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.548874    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:06:15.548921 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.548904    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:06:15.620794 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.620762    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" event={"ID":"a7ae35a9-7c97-4620-bf14-ef380e2090fd","Type":"ContainerStarted","Data":"48fcc33b24b9538b477f31b8d60147a2e232593756af5958e9b3724804287eda"}
Nov 03 20:06:15.620909 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.620828    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" event={"ID":"a7ae35a9-7c97-4620-bf14-ef380e2090fd","Type":"ContainerStarted","Data":"86d43d55f69fef80c4c22a9f78e6877e42943a3e6195c403bd6e1ddc8d77bbef"}
Nov 03 20:06:15.647749 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:15.647692    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" podStartSLOduration=1.647674407 podStartE2EDuration="1.647674407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:06:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:06:15.644361181 +0000 UTC m=+2376.552383121" watchObservedRunningTime="2026-11-03 20:06:15.647674407 +0000 UTC m=+2376.555696346"
Nov 03 20:06:16.266837 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:16.266800    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:16.272134 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:16.272107    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:16.626318 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:16.626242    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:16.627124 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:16.627101    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:19.049402 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.049362    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"]
Nov 03 20:06:19.053986 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.053964    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.066190 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.066158    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"]
Nov 03 20:06:19.078887 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.078860    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-66e889-serving-cert\""
Nov 03 20:06:19.197363 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.197331    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-metrics-certs\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.197542 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.197388    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ng5n\" (UniqueName: \"kubernetes.io/projected/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-kube-api-access-4ng5n\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.197542 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.197481    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.197542 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.197499    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.299026 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.298990    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-metrics-certs\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.299248 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.299173    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ng5n\" (UniqueName: \"kubernetes.io/projected/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-kube-api-access-4ng5n\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.299248 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.299222    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.299387 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.299248    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.299955 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.299893    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.301575 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.301547    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-metrics-certs\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.301661 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.301652    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.311720 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.311695    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ng5n\" (UniqueName: \"kubernetes.io/projected/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-kube-api-access-4ng5n\") pod \"maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.374215 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.374192    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:19.648532 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:19.648509    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"]
Nov 03 20:06:19.650286 ip-10-0-143-212 kubenswrapper[5051]: W1103 20:06:19.650252    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c1995bf_ab76_4b0b_9ec0_83aaba57b096.slice/crio-5353358ad5e1ccc585a69becbeb5c71f3ab61efd71ec6ea6059a4e8e1dd1926a WatchSource:0}: Error finding container 5353358ad5e1ccc585a69becbeb5c71f3ab61efd71ec6ea6059a4e8e1dd1926a: Status 404 returned error can't find the container with id 5353358ad5e1ccc585a69becbeb5c71f3ab61efd71ec6ea6059a4e8e1dd1926a
Nov 03 20:06:20.649563 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:20.649528    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4" event={"ID":"7c1995bf-ab76-4b0b-9ec0-83aaba57b096","Type":"ContainerStarted","Data":"2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3"}
Nov 03 20:06:20.650093 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:20.649587    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4" event={"ID":"7c1995bf-ab76-4b0b-9ec0-83aaba57b096","Type":"ContainerStarted","Data":"5353358ad5e1ccc585a69becbeb5c71f3ab61efd71ec6ea6059a4e8e1dd1926a"}
Nov 03 20:06:20.650093 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:20.649628    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:20.670559 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:20.670515    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4" podStartSLOduration=1.181496327 podStartE2EDuration="1.670502093s" totalImagesPullingTime="489.005766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:06:20.668917903 +0000 UTC m=+2381.576939842" watchObservedRunningTime="2026-11-03 20:06:20.670502093 +0000 UTC m=+2381.578524031"
Nov 03 20:06:21.309120 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:21.309084    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:06:26.672633 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:26.672602    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:06:31.627650 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:31.627618    5051 patch_prober.go:29] interesting pod/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 03 20:06:31.628080 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:31.627669    5051 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 03 20:06:31.722271 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:31.722243    5051 generic.go:356] "Generic (PLEG): container finished" podID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerID="48fcc33b24b9538b477f31b8d60147a2e232593756af5958e9b3724804287eda" exitCode=0
Nov 03 20:06:31.722443 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:31.722315    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" event={"ID":"a7ae35a9-7c97-4620-bf14-ef380e2090fd","Type":"ContainerDied","Data":"48fcc33b24b9538b477f31b8d60147a2e232593756af5958e9b3724804287eda"}
Nov 03 20:06:31.722758 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:31.722744    5051 scope.go:122] "RemoveContainer" containerID="48fcc33b24b9538b477f31b8d60147a2e232593756af5958e9b3724804287eda"
Nov 03 20:06:31.723978 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:31.723949    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:06:31.724093 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:31.724018    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:06:31.724093 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:31.724066    5051 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 03 20:06:32.729782 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:32.729748    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" event={"ID":"a7ae35a9-7c97-4620-bf14-ef380e2090fd","Type":"ContainerStarted","Data":"3e3a6ba5d7e195ec686c5ec849583113174d258f7845e9632efb959489def003"}
Nov 03 20:06:32.795272 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:06:32.795233    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:06:33.266863 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:33.266823    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:33.267112 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:33.267068    5051 patch_prober.go:29] interesting pod/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 03 20:06:33.267210 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:33.267124    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 03 20:06:34.267454 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:34.267414    5051 patch_prober.go:29] interesting pod/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 03 20:06:34.267937 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:34.267480    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 03 20:06:35.267284 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:35.267236    5051 patch_prober.go:29] interesting pod/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 03 20:06:35.267468 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:35.267319    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 03 20:06:36.267677 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:36.267636    5051 patch_prober.go:29] interesting pod/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 03 20:06:36.268098 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:36.267721    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 03 20:06:37.267744 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:37.267709    5051 patch_prober.go:29] interesting pod/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused" start-of-body=
Nov 03 20:06:37.268199 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:37.267774    5051 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.152:15021/healthz/ready\": dial tcp 10.132.0.152:15021: connect: connection refused"
Nov 03 20:06:38.271701 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:38.271673    5051 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:38.272150 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:38.272033    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:38.272638 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:38.272614    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:06:40.551911 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:40.551880    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 20:06:40.674194 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:06:40.674170    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 20:07:02.766892 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:07:02.766854    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:07:32.767278 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:07:32.767243    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:08:02.767121 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:08:02.767082    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:08:08.983098 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:08.983060    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"]
Nov 03 20:08:08.983729 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:08.983322    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4" podUID="7c1995bf-ab76-4b0b-9ec0-83aaba57b096" containerName="maas-api" containerID="cri-o://2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3" gracePeriod=30
Nov 03 20:08:09.271068 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.270903    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:08:09.271623 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.271593    5051 generic.go:356] "Generic (PLEG): container finished" podID="7c1995bf-ab76-4b0b-9ec0-83aaba57b096" containerID="2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3" exitCode=0
Nov 03 20:08:09.271812 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.271668    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4" event={"ID":"7c1995bf-ab76-4b0b-9ec0-83aaba57b096","Type":"ContainerDied","Data":"2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3"}
Nov 03 20:08:09.271812 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.271713    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4" event={"ID":"7c1995bf-ab76-4b0b-9ec0-83aaba57b096","Type":"ContainerDied","Data":"5353358ad5e1ccc585a69becbeb5c71f3ab61efd71ec6ea6059a4e8e1dd1926a"}
Nov 03 20:08:09.271812 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.271736    5051 scope.go:122] "RemoveContainer" containerID="2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3"
Nov 03 20:08:09.282268 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.282248    5051 scope.go:122] "RemoveContainer" containerID="2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3"
Nov 03 20:08:09.282589 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:08:09.282562    5051 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3\": container with ID starting with 2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3 not found: ID does not exist" containerID="2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3"
Nov 03 20:08:09.282669 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.282611    5051 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3"} err="failed to get container status \"2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3\": rpc error: code = NotFound desc = could not find container \"2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3\": container with ID starting with 2afd269b82e014fd1c53080dfd1f9a98dcbe4327bc886cf87c2c7cc62b041cb3 not found: ID does not exist"
Nov 03 20:08:09.417694 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.417660    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-metrics-certs\") pod \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") "
Nov 03 20:08:09.417861 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.417727    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-maas-api-tls\") pod \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") "
Nov 03 20:08:09.417861 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.417808    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-kube-api-access-4ng5n\" (UniqueName: \"kubernetes.io/projected/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-kube-api-access-4ng5n\") pod \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") "
Nov 03 20:08:09.417861 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.417849    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-openshift-service-ca\") pod \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\" (UID: \"7c1995bf-ab76-4b0b-9ec0-83aaba57b096\") "
Nov 03 20:08:09.418285 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.418259    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-openshift-service-ca" pod "7c1995bf-ab76-4b0b-9ec0-83aaba57b096" (UID: "7c1995bf-ab76-4b0b-9ec0-83aaba57b096"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:08:09.419902 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.419874    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-kube-api-access-4ng5n" pod "7c1995bf-ab76-4b0b-9ec0-83aaba57b096" (UID: "7c1995bf-ab76-4b0b-9ec0-83aaba57b096"). InnerVolumeSpecName "kube-api-access-4ng5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:08:09.420275 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.420248    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-metrics-certs" pod "7c1995bf-ab76-4b0b-9ec0-83aaba57b096" (UID: "7c1995bf-ab76-4b0b-9ec0-83aaba57b096"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:08:09.420275 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.420270    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-maas-api-tls" pod "7c1995bf-ab76-4b0b-9ec0-83aaba57b096" (UID: "7c1995bf-ab76-4b0b-9ec0-83aaba57b096"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:08:09.519387 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.519323    5051 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-maas-api-tls\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:09.519387 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.519347    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ng5n\" (UniqueName: \"kubernetes.io/projected/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-kube-api-access-4ng5n\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:09.519387 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.519359    5051 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-openshift-service-ca\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:09.519387 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:09.519367    5051 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c1995bf-ab76-4b0b-9ec0-83aaba57b096-metrics-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:10.278111 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:10.278081    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"
Nov 03 20:08:10.302853 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:10.302827    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"]
Nov 03 20:08:10.305742 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:10.305720    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-66e889-5c586db9fd-c8dt4"]
Nov 03 20:08:10.960348 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:10.960303    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:08:11.228301 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:11.228220    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c1995bf-ab76-4b0b-9ec0-83aaba57b096" path="/var/lib/kubelet/pods/7c1995bf-ab76-4b0b-9ec0-83aaba57b096/volumes"
Nov 03 20:08:16.178438 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:16.178402    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"]
Nov 03 20:08:16.178966 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:16.178640    5051 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy" containerID="cri-o://3e3a6ba5d7e195ec686c5ec849583113174d258f7845e9632efb959489def003" gracePeriod=30
Nov 03 20:08:18.397732 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:18.397697    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:08:21.337879 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.337845    5051 generic.go:356] "Generic (PLEG): container finished" podID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerID="3e3a6ba5d7e195ec686c5ec849583113174d258f7845e9632efb959489def003" exitCode=0
Nov 03 20:08:21.338300 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.337891    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" event={"ID":"a7ae35a9-7c97-4620-bf14-ef380e2090fd","Type":"ContainerDied","Data":"3e3a6ba5d7e195ec686c5ec849583113174d258f7845e9632efb959489def003"}
Nov 03 20:08:21.338300 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.337937    5051 scope.go:122] "RemoveContainer" containerID="48fcc33b24b9538b477f31b8d60147a2e232593756af5958e9b3724804287eda"
Nov 03 20:08:21.482648 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.482622    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:08:21.637326 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.637298    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-certs\") pod \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") "
Nov 03 20:08:21.637326 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.637332    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-data\") pod \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") "
Nov 03 20:08:21.637561 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.637354    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-podinfo\") pod \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") "
Nov 03 20:08:21.637561 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.637374    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istiod-ca-cert\") pod \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") "
Nov 03 20:08:21.637561 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.637394    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-token\" (UniqueName: \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-token\") pod \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") "
Nov 03 20:08:21.637561 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.637441    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-socket\") pod \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") "
Nov 03 20:08:21.637769 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.637583    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-kube-api-access-zdscc\" (UniqueName: \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-kube-api-access-zdscc\") pod \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") "
Nov 03 20:08:21.637769 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.637637    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-data" pod "a7ae35a9-7c97-4620-bf14-ef380e2090fd" (UID: "a7ae35a9-7c97-4620-bf14-ef380e2090fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:08:21.637769 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.637712    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-certs" pod "a7ae35a9-7c97-4620-bf14-ef380e2090fd" (UID: "a7ae35a9-7c97-4620-bf14-ef380e2090fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:08:21.637943 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.637826    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-socket" pod "a7ae35a9-7c97-4620-bf14-ef380e2090fd" (UID: "a7ae35a9-7c97-4620-bf14-ef380e2090fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:08:21.637943 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.637868    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istiod-ca-cert" pod "a7ae35a9-7c97-4620-bf14-ef380e2090fd" (UID: "a7ae35a9-7c97-4620-bf14-ef380e2090fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:08:21.638070 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.638023    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-credential-socket\") pod \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") "
Nov 03 20:08:21.638163 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.638144    5051 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-envoy\") pod \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\" (UID: \"a7ae35a9-7c97-4620-bf14-ef380e2090fd\") "
Nov 03 20:08:21.638293 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.638261    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-credential-socket" pod "a7ae35a9-7c97-4620-bf14-ef380e2090fd" (UID: "a7ae35a9-7c97-4620-bf14-ef380e2090fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:08:21.638570 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.638547    5051 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:21.638649 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.638576    5051 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-credential-socket\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:21.638649 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.638600    5051 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-workload-certs\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:21.638649 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.638615    5051 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-data\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:21.638649 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.638629    5051 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istiod-ca-cert\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:21.639743 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.639720    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-podinfo" pod "a7ae35a9-7c97-4620-bf14-ef380e2090fd" (UID: "a7ae35a9-7c97-4620-bf14-ef380e2090fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 20:08:21.639743 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.639723    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-kube-api-access-zdscc" pod "a7ae35a9-7c97-4620-bf14-ef380e2090fd" (UID: "a7ae35a9-7c97-4620-bf14-ef380e2090fd"). InnerVolumeSpecName "kube-api-access-zdscc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:08:21.639924 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.639835    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-token" pod "a7ae35a9-7c97-4620-bf14-ef380e2090fd" (UID: "a7ae35a9-7c97-4620-bf14-ef380e2090fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:08:21.640197 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.640182    5051 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-envoy" pod "a7ae35a9-7c97-4620-bf14-ef380e2090fd" (UID: "a7ae35a9-7c97-4620-bf14-ef380e2090fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:08:21.739422 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.739393    5051 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-envoy\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:21.739422 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.739419    5051 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-podinfo\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:21.739600 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.739430    5051 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-istio-token\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:21.739600 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:21.739439    5051 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdscc\" (UniqueName: \"kubernetes.io/projected/a7ae35a9-7c97-4620-bf14-ef380e2090fd-kube-api-access-zdscc\") on node \"ip-10-0-143-212.us-east-2.compute.internal\" DevicePath \"\""
Nov 03 20:08:22.344948 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:22.344915    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59" event={"ID":"a7ae35a9-7c97-4620-bf14-ef380e2090fd","Type":"ContainerDied","Data":"86d43d55f69fef80c4c22a9f78e6877e42943a3e6195c403bd6e1ddc8d77bbef"}
Nov 03 20:08:22.344948 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:22.344939    5051 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"
Nov 03 20:08:22.345442 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:22.344975    5051 scope.go:122] "RemoveContainer" containerID="3e3a6ba5d7e195ec686c5ec849583113174d258f7845e9632efb959489def003"
Nov 03 20:08:22.368340 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:22.368317    5051 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"]
Nov 03 20:08:22.372289 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:22.372263    5051 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-66e889-openshift-default-854b995c4f-mzj59"]
Nov 03 20:08:23.229469 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:23.229433    5051 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" path="/var/lib/kubelet/pods/a7ae35a9-7c97-4620-bf14-ef380e2090fd/volumes"
Nov 03 20:08:26.870151 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:26.870111    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:08:32.766691 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:08:32.766652    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:08:42.302756 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:08:42.302717    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:09:02.767579 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:09:02.767539    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:09:09.614511 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:09.614476    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2cwgg"]
Nov 03 20:09:21.223805 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:21.223698    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-f47bcb7b4-ftdw4_5ae94d23-3b48-4bf4-bc45-d4e248468c0f/manager/0.log"
Nov 03 20:09:21.428326 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:21.428294    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-z87sc_812da5c0-5a4f-4013-b799-bf5e410f49a2/manager/0.log"
Nov 03 20:09:22.528722 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:22.528686    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-vnbhz_745c0adf-f3a5-4eb7-adfd-c1d4ebb72806/manager/0.log"
Nov 03 20:09:23.740364 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:23.740334    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-755bc5f869-stjv7_bb932f93-3367-4339-95d0-6408d13a3cf8/maas-api/0.log"
Nov 03 20:09:27.722386 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:27.722350    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-2cwgg_77139ff2-796c-4caa-bd8b-e78bc69db3e6/limitador/0.log"
Nov 03 20:09:29.167703 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:29.167669    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b4fd54d5-44frg_292c8c65-8eb5-45ce-b09b-6b235e2e2c9c/kube-auth-proxy/0.log"
Nov 03 20:09:29.921793 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:29.921760    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-c684f_b25cf9ad-db43-42cb-8fc8-883c249e8796/istio-proxy/0.log"
Nov 03 20:09:30.945138 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:30.945102    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-qtcsx_159b05c6-8444-4ed2-95d7-7d07dd9d8b34/istio-proxy/0.log"
Nov 03 20:09:31.339123 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:31.339059    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-xtsbv_9b1113b4-950d-4cf7-a856-1290aa4a1934/istio-proxy/0.log"
Nov 03 20:09:32.092323 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:32.092295    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-687ff884df-rq9lk_b638dbdf-a1bf-4271-be87-4937d20d4a61/router/0.log"
Nov 03 20:09:32.767333 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:09:32.767295    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:09:33.133187 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:33.133113    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n_c8f9579a-928c-41e0-a4ad-3438618bae19/main/0.log"
Nov 03 20:09:33.517807 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:33.517737    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-v4fqs_e4d4d3cd-6773-4b3b-80c6-52f5f9af92da/main/0.log"
Nov 03 20:09:33.901242 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:33.901155    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf_9e2957a8-5aed-4931-a478-eadcd6eaa562/main/0.log"
Nov 03 20:09:34.284433 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:34.284385    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8_1af7e503-62a3-4481-9c70-b24c8313c154/main/0.log"
Nov 03 20:09:45.018506 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:45.018478    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-f47bcb7b4-ftdw4_5ae94d23-3b48-4bf4-bc45-d4e248468c0f/manager/0.log"
Nov 03 20:09:45.223052 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:45.223017    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-z87sc_812da5c0-5a4f-4013-b799-bf5e410f49a2/manager/0.log"
Nov 03 20:09:46.306017 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:46.305982    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-vnbhz_745c0adf-f3a5-4eb7-adfd-c1d4ebb72806/manager/0.log"
Nov 03 20:09:47.325921 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:47.325893    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-755bc5f869-stjv7_bb932f93-3367-4339-95d0-6408d13a3cf8/maas-api/0.log"
Nov 03 20:09:50.996373 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:50.996343    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-2cwgg_77139ff2-796c-4caa-bd8b-e78bc69db3e6/limitador/0.log"
Nov 03 20:09:52.189751 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:52.189716    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b4fd54d5-44frg_292c8c65-8eb5-45ce-b09b-6b235e2e2c9c/kube-auth-proxy/0.log"
Nov 03 20:09:52.935796 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:52.935769    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-c684f_b25cf9ad-db43-42cb-8fc8-883c249e8796/istio-proxy/0.log"
Nov 03 20:09:53.872372 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:53.872327    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-qtcsx_159b05c6-8444-4ed2-95d7-7d07dd9d8b34/istio-proxy/0.log"
Nov 03 20:09:54.263439 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:54.263401    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-xtsbv_9b1113b4-950d-4cf7-a856-1290aa4a1934/istio-proxy/0.log"
Nov 03 20:09:55.028485 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:55.028457    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-687ff884df-rq9lk_b638dbdf-a1bf-4271-be87-4937d20d4a61/router/0.log"
Nov 03 20:09:55.578489 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:55.578460    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-qbc8n_c8f9579a-928c-41e0-a4ad-3438618bae19/main/0.log"
Nov 03 20:09:55.936501 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:55.936423    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-v4fqs_e4d4d3cd-6773-4b3b-80c6-52f5f9af92da/main/0.log"
Nov 03 20:09:56.298739 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:56.298706    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b885ndf_9e2957a8-5aed-4931-a478-eadcd6eaa562/main/0.log"
Nov 03 20:09:56.680913 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:09:56.680823    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-fcbn8_1af7e503-62a3-4481-9c70-b24c8313c154/main/0.log"
Nov 03 20:10:02.766027 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:10:02.765993    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:10:32.767023 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:10:32.766985    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:11:02.766291 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:11:02.766256    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:11:23.624518 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.624483    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b8lpv/must-gather-j4pt9"]
Nov 03 20:11:23.625036 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.624963    5051 cpu_manager.go:414] "containerMap: removing container" podUID="7c1995bf-ab76-4b0b-9ec0-83aaba57b096" containerName="maas-api"
Nov 03 20:11:23.625036 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.624976    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c1995bf-ab76-4b0b-9ec0-83aaba57b096" containerName="maas-api"
Nov 03 20:11:23.625036 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.624987    5051 cpu_manager.go:414] "containerMap: removing container" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy"
Nov 03 20:11:23.625036 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.624992    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy"
Nov 03 20:11:23.625036 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.625004    5051 cpu_manager.go:414] "containerMap: removing container" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy"
Nov 03 20:11:23.625036 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.625012    5051 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy"
Nov 03 20:11:23.625283 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.625114    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c1995bf-ab76-4b0b-9ec0-83aaba57b096" containerName="maas-api"
Nov 03 20:11:23.625283 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.625124    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy"
Nov 03 20:11:23.625283 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.625131    5051 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7ae35a9-7c97-4620-bf14-ef380e2090fd" containerName="istio-proxy"
Nov 03 20:11:23.628673 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.628653    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b8lpv/must-gather-j4pt9"
Nov 03 20:11:23.632837 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.632816    5051 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b8lpv\"/\"default-dockercfg-tjpxw\""
Nov 03 20:11:23.634394 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.634371    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b8lpv\"/\"openshift-service-ca.crt\""
Nov 03 20:11:23.634530 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.634402    5051 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b8lpv\"/\"kube-root-ca.crt\""
Nov 03 20:11:23.647302 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.647283    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b8lpv/must-gather-j4pt9"]
Nov 03 20:11:23.731696 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.731657    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a4afc850-2ad3-4880-b53b-29633555c23e-must-gather-output\") pod \"must-gather-j4pt9\" (UID: \"a4afc850-2ad3-4880-b53b-29633555c23e\") " pod="openshift-must-gather-b8lpv/must-gather-j4pt9"
Nov 03 20:11:23.731856 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.731723    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbbc5\" (UniqueName: \"kubernetes.io/projected/a4afc850-2ad3-4880-b53b-29633555c23e-kube-api-access-fbbc5\") pod \"must-gather-j4pt9\" (UID: \"a4afc850-2ad3-4880-b53b-29633555c23e\") " pod="openshift-must-gather-b8lpv/must-gather-j4pt9"
Nov 03 20:11:23.832469 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.832438    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbbc5\" (UniqueName: \"kubernetes.io/projected/a4afc850-2ad3-4880-b53b-29633555c23e-kube-api-access-fbbc5\") pod \"must-gather-j4pt9\" (UID: \"a4afc850-2ad3-4880-b53b-29633555c23e\") " pod="openshift-must-gather-b8lpv/must-gather-j4pt9"
Nov 03 20:11:23.832585 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.832568    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a4afc850-2ad3-4880-b53b-29633555c23e-must-gather-output\") pod \"must-gather-j4pt9\" (UID: \"a4afc850-2ad3-4880-b53b-29633555c23e\") " pod="openshift-must-gather-b8lpv/must-gather-j4pt9"
Nov 03 20:11:23.832909 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.832893    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a4afc850-2ad3-4880-b53b-29633555c23e-must-gather-output\") pod \"must-gather-j4pt9\" (UID: \"a4afc850-2ad3-4880-b53b-29633555c23e\") " pod="openshift-must-gather-b8lpv/must-gather-j4pt9"
Nov 03 20:11:23.845462 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.845439    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbbc5\" (UniqueName: \"kubernetes.io/projected/a4afc850-2ad3-4880-b53b-29633555c23e-kube-api-access-fbbc5\") pod \"must-gather-j4pt9\" (UID: \"a4afc850-2ad3-4880-b53b-29633555c23e\") " pod="openshift-must-gather-b8lpv/must-gather-j4pt9"
Nov 03 20:11:23.943110 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:23.943028    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b8lpv/must-gather-j4pt9"
Nov 03 20:11:24.213998 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:24.213972    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b8lpv/must-gather-j4pt9"]
Nov 03 20:11:24.215713 ip-10-0-143-212 kubenswrapper[5051]: W1103 20:11:24.215688    5051 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4afc850_2ad3_4880_b53b_29633555c23e.slice/crio-f949c17661dcfb7799741fe73522c0e9ccf52cd1e4dcbb72838f95dfbd12b38c WatchSource:0}: Error finding container f949c17661dcfb7799741fe73522c0e9ccf52cd1e4dcbb72838f95dfbd12b38c: Status 404 returned error can't find the container with id f949c17661dcfb7799741fe73522c0e9ccf52cd1e4dcbb72838f95dfbd12b38c
Nov 03 20:11:24.217788 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:24.217774    5051 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 20:11:24.336885 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:24.336854    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b8lpv/must-gather-j4pt9" event={"ID":"a4afc850-2ad3-4880-b53b-29633555c23e","Type":"ContainerStarted","Data":"f949c17661dcfb7799741fe73522c0e9ccf52cd1e4dcbb72838f95dfbd12b38c"}
Nov 03 20:11:25.345473 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:25.344799    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b8lpv/must-gather-j4pt9" event={"ID":"a4afc850-2ad3-4880-b53b-29633555c23e","Type":"ContainerStarted","Data":"508d70218b5091876f3f9130b9baf2dd58238dbb562205b033da67e9f6b2ffeb"}
Nov 03 20:11:25.345473 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:25.344858    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b8lpv/must-gather-j4pt9" event={"ID":"a4afc850-2ad3-4880-b53b-29633555c23e","Type":"ContainerStarted","Data":"67490ac68fc43eb15a7fbb3deed6860418371b079515342eb153f34d4170293e"}
Nov 03 20:11:25.362595 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:25.362528    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-b8lpv/must-gather-j4pt9" podStartSLOduration=1.578838947 podStartE2EDuration="2.362509254s" totalImagesPullingTime="783.670307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:11:25.362061509 +0000 UTC m=+2686.270083449" watchObservedRunningTime="2026-11-03 20:11:25.362509254 +0000 UTC m=+2686.270531195"
Nov 03 20:11:26.903866 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:26.903830    5051 ???:1] "http: TLS handshake error from 10.0.143.212:45242: EOF"
Nov 03 20:11:26.904451 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:26.904414    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-txdf4_074a9efa-d51e-4192-841c-3cd308d89945/global-pull-secret-syncer/0.log"
Nov 03 20:11:26.961764 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:26.961735    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-9cxfx_8529580b-2186-4766-99d3-b55374b668ef/connection-checker/0.log"
Nov 03 20:11:26.990987 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:26.990963    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-ctdfb_bf43b3af-6048-4d1b-9a8c-96efd387cfc4/connection-checker/0.log"
Nov 03 20:11:27.020412 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:27.020383    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-pxn4t_563541bc-4a0f-4e6e-b6f0-ebd9264f537d/connection-checker/0.log"
Nov 03 20:11:27.127996 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:27.127958    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8hvrb_4fd1c83c-4a87-43b1-b25e-8b9b23922aab/konnectivity-agent/0.log"
Nov 03 20:11:27.265592 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:27.265556    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-212.us-east-2.compute.internal_131c0f9830433abeb8e4c85b17b38230/haproxy/0.log"
Nov 03 20:11:32.799309 ip-10-0-143-212 kubenswrapper[5051]: E1103 20:11:32.799265    5051 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lb4p7" containerName="extractor"
Nov 03 20:11:33.589065 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:33.587914    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-2cwgg_77139ff2-796c-4caa-bd8b-e78bc69db3e6/limitador/0.log"
Nov 03 20:11:35.131680 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:35.131643    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-x8xpg_13fdaf11-64d1-445d-b055-fbec6b90c1bb/cluster-monitoring-operator/0.log"
Nov 03 20:11:35.432953 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:35.432870    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ghjhp_78bddb58-2426-4956-a2a2-c5bc92df3c93/node-exporter/0.log"
Nov 03 20:11:35.459928 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:35.459894    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ghjhp_78bddb58-2426-4956-a2a2-c5bc92df3c93/kube-rbac-proxy/0.log"
Nov 03 20:11:35.488612 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:35.488582    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ghjhp_78bddb58-2426-4956-a2a2-c5bc92df3c93/init-textfile/0.log"
Nov 03 20:11:35.950456 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:35.950430    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-shqbr_611c335f-c556-40e4-9c41-110d9b511ff1/prometheus-operator/0.log"
Nov 03 20:11:35.974450 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:35.974423    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-shqbr_611c335f-c556-40e4-9c41-110d9b511ff1/kube-rbac-proxy/0.log"
Nov 03 20:11:36.011655 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.011621    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-4d2p6_d0cd4ad5-f51c-426b-912a-7a740a19879e/prometheus-operator-admission-webhook/0.log"
Nov 03 20:11:36.793078 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.790542    5051 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"]
Nov 03 20:11:36.803081 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.802785    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"]
Nov 03 20:11:36.803081 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.802940    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.837328 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.837296    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-sys\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.837590 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.837567    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-859jq\" (UniqueName: \"kubernetes.io/projected/a8d87920-4893-43c0-9baf-79bdde652bf2-kube-api-access-859jq\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.837738 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.837720    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-lib-modules\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.837873 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.837856    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-proc\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.838010 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.837996    5051 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-podres\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.939405 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.939359    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-sys\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.939618 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.939438    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-859jq\" (UniqueName: \"kubernetes.io/projected/a8d87920-4893-43c0-9baf-79bdde652bf2-kube-api-access-859jq\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.939618 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.939471    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-sys\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.939618 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.939480    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-lib-modules\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.939618 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.939549    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-lib-modules\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.939618 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.939553    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-proc\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.939618 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.939602    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-proc\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.939921 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.939658    5051 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-podres\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.939921 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.939818    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8d87920-4893-43c0-9baf-79bdde652bf2-podres\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:36.954324 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:36.954290    5051 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-859jq\" (UniqueName: \"kubernetes.io/projected/a8d87920-4893-43c0-9baf-79bdde652bf2-kube-api-access-859jq\") pod \"perf-node-gather-daemonset-ld67v\" (UID: \"a8d87920-4893-43c0-9baf-79bdde652bf2\") " pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:37.122010 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:37.121925    5051 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:37.422280 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:37.417207    5051 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"]
Nov 03 20:11:37.463256 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:37.463229    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v" event={"ID":"a8d87920-4893-43c0-9baf-79bdde652bf2","Type":"ContainerStarted","Data":"19fa30914772a8aa5824750fa737f56a6e64c28c07ef97b2c888439ad2aa8ddd"}
Nov 03 20:11:37.607962 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:37.607922    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-pv5k4_51919e3c-99a2-484b-96ac-35948a9eec74/networking-console-plugin/0.log"
Nov 03 20:11:38.299280 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:38.299255    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 20:11:38.304737 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:38.304709    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/3.log"
Nov 03 20:11:38.471842 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:38.471804    5051 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v" event={"ID":"a8d87920-4893-43c0-9baf-79bdde652bf2","Type":"ContainerStarted","Data":"9019ee610ea9d70a32d16cfb00e68270590dc9bf11169d7f955c1a91186d7228"}
Nov 03 20:11:38.472033 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:38.471965    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:38.490743 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:38.490681    5051 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v" podStartSLOduration=2.4906694959999998 podStartE2EDuration="2.490669496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:11:38.490409189 +0000 UTC m=+2699.398431127" watchObservedRunningTime="2026-11-03 20:11:38.490669496 +0000 UTC m=+2699.398691426"
Nov 03 20:11:38.812636 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:38.812587    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-77c78ddc57-lbkk9_81685b9f-a697-4a2d-94ba-b5364c9c6c1c/console/0.log"
Nov 03 20:11:38.851959 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:38.851932    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-r4rnd_7f2a8a7d-2f45-4ce6-87be-08adffed4ac1/download-server/0.log"
Nov 03 20:11:39.484279 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:39.484251    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-4gdd9_dd1caca1-f187-4c53-860b-5443c3b95924/volume-data-source-validator/0.log"
Nov 03 20:11:40.594035 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:40.593998    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 20:11:40.709425 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:40.709397    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-hb5c6_c5f64340-0f40-47b9-949c-6f25f4ef05ab/console-operator/2.log"
Nov 03 20:11:40.746693 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:40.746669    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nlt2r_10683ad8-f14a-4034-9768-bb8b168141d1/dns/0.log"
Nov 03 20:11:40.775514 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:40.775489    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nlt2r_10683ad8-f14a-4034-9768-bb8b168141d1/kube-rbac-proxy/0.log"
Nov 03 20:11:40.863746 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:40.863679    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rbszw_61c81f40-4aae-4c97-9be9-30da0effd8cb/dns-node-resolver/0.log"
Nov 03 20:11:41.543848 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:41.543815    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-66b954b64f-9znvn_9874f516-0f3d-4220-9b89-421bd65938e6/registry/0.log"
Nov 03 20:11:41.629036 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:41.628999    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ss5gj_ca7d9994-2fc4-4328-9c50-918954121ead/node-ca/0.log"
Nov 03 20:11:44.490658 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:44.490627    5051 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b8lpv/perf-node-gather-daemonset-ld67v"
Nov 03 20:11:45.304061 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:45.303718    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b4fd54d5-44frg_292c8c65-8eb5-45ce-b09b-6b235e2e2c9c/kube-auth-proxy/0.log"
Nov 03 20:11:45.449459 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:45.449422    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-c684f_b25cf9ad-db43-42cb-8fc8-883c249e8796/istio-proxy/0.log"
Nov 03 20:11:45.631982 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:45.631893    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-qtcsx_159b05c6-8444-4ed2-95d7-7d07dd9d8b34/istio-proxy/0.log"
Nov 03 20:11:45.740500 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:45.740471    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-xtsbv_9b1113b4-950d-4cf7-a856-1290aa4a1934/istio-proxy/0.log"
Nov 03 20:11:45.952538 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:45.952456    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-687ff884df-rq9lk_b638dbdf-a1bf-4271-be87-4937d20d4a61/router/0.log"
Nov 03 20:11:47.823851 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:47.823813    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g9g77_4d145360-58fb-4376-8126-7933a7abaae2/serve-healthcheck-canary/0.log"
Nov 03 20:11:48.467897 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:48.467846    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-2qjr4_c4f3fa56-4b85-46bd-8c97-94da036ce8bb/insights-operator/0.log"
Nov 03 20:11:48.468758 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:48.468699    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-2qjr4_c4f3fa56-4b85-46bd-8c97-94da036ce8bb/insights-operator/1.log"
Nov 03 20:11:48.590622 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:48.590591    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lb4p7_3ae31de6-e744-4e9a-a9dd-7c5203acad17/kube-rbac-proxy/0.log"
Nov 03 20:11:48.620486 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:48.620453    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lb4p7_3ae31de6-e744-4e9a-a9dd-7c5203acad17/exporter/0.log"
Nov 03 20:11:48.651797 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:48.651767    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lb4p7_3ae31de6-e744-4e9a-a9dd-7c5203acad17/extractor/0.log"
Nov 03 20:11:48.793210 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:48.793141    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_periodic-gathering-tddrc-xp7ln_474276d6-b0b6-4ac1-864a-1c57a6465bc0/insights-gathering/0.log"
Nov 03 20:11:49.244381 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:49.244338    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b4fd54d5-44frg_292c8c65-8eb5-45ce-b09b-6b235e2e2c9c/kube-auth-proxy/0.log"
Nov 03 20:11:49.330290 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:49.330265    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-c684f_b25cf9ad-db43-42cb-8fc8-883c249e8796/istio-proxy/0.log"
Nov 03 20:11:49.430628 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:49.430594    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-qtcsx_159b05c6-8444-4ed2-95d7-7d07dd9d8b34/istio-proxy/0.log"
Nov 03 20:11:49.471011 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:49.470984    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-xtsbv_9b1113b4-950d-4cf7-a856-1290aa4a1934/istio-proxy/0.log"
Nov 03 20:11:49.545396 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:49.545319    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-687ff884df-rq9lk_b638dbdf-a1bf-4271-be87-4937d20d4a61/router/0.log"
Nov 03 20:11:51.270813 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:51.270784    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-7646b49899-mn6px_cc111e19-92b3-4aea-b94f-b7fe3adb5f22/manager/0.log"
Nov 03 20:11:51.301377 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:51.301343    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-7646b49899-mn6px_cc111e19-92b3-4aea-b94f-b7fe3adb5f22/copy-manifests/0.log"
Nov 03 20:11:52.603359 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:52.603325    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-operator-674cd59dc5-rlbrl_dc80a989-407b-4129-8954-a9d3c8249aae/jobset-operator/0.log"
Nov 03 20:11:53.392095 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:53.392034    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-f47bcb7b4-ftdw4_5ae94d23-3b48-4bf4-bc45-d4e248468c0f/manager/0.log"
Nov 03 20:11:53.420786 ip-10-0-143-212 kubenswrapper[5051]: I1103 20:11:53.420763    5051 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-z87sc_812da5c0-5a4f-4013-b799-bf5e410f49a2/manager/0.log"